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FOREWORD 


Over  the  years,  scientists  have  developed  many  techniques  for  extracting 
and  evaluating  Information  from  experimental  data.  One  of  the  reasons 
for  holding  the  design  conferences  Is  to  develop,  In  Amy  scientists, 
an  appreciation  for  and  the  necessary  skills  to  handle  these  techniques. 

A special  feature  associated  with  the  meeting  this  year  was  organized 
to  help  develop  some  of  these  skills.  This  was  a basic  tutorial  seminar 
entitled,  "Introduction  to  the  Fundamentals  of  Experimental  Design" 
given  on  2-3  October  1978  by  Dr.  George  E.  P.  Box,  the  Ronald  Alymer 
Fisher  Professor  of  Statistics  at  the  University  of  Wisconsin  and 
the  Mathematics  Research  Center.  This  course  was  designed  for  engineers 
and  other  scientists  with  little  or  no  formal  training  In  statistics, 
and  who  are  Involved  In  generating  physical  measurements  from  experiments. 

It  presented  basic  notions  and  statistical  techniques  which  allows  one  to 
minimize  data  variance  or  variability,  and  hence  ultimately  enhances  the 
opportunities  for  recovering  data  Information  In  later  analyses.  The  forty 
or  so  Army  scientists  attending  this  course  were  prepared  to  comprehend 
the  papers  given  at  the  Twenty-fourth  Conference  on  the  Design  of  Experiments 
In  Army  Research,  Development  and  Testing. 


Members  of  the  Program  Committee  for  this  conference  were  pleased  to  obtain 
the  services  of  the  following  Invited  speakers  to  talk  on  topics  of  current 
interest  to  Army  personnel. 


Speaker  and  Institute 

Professor  Norman  Draper 
University  of  Wisconsin-Madlson 

Professor  Ralph  Bradley 
Florida  State  University 

Professor  Grace  Wahba 
University  of  Wisconsin-Madlson 


Title  of  Address 
RIDGE  REGRESSION 


STATISTICAL  ANALYSIS  OF  WEATHER 
MODIFICATION  EXPERIMENTS 

DESIGN  PROBLEMS  IN  RECOVERING 
FUNCTIONS  OF  TWO  OR  SEVERAL 
VARIABLES 


111 


STATISTICAL  CONSULTING  ' 

i 

RECENT  ADVANCES  IN  GRAPHICAL  | 

TECHNIQUES  FOR  ANALYZING  FAILURE 
DATA  j 

In  addition  to  tha  Invited  addresses*  there  were  nineteen  contributed  papers.  j 

Many  of  these  Informative  talks  covered  areas  associated  with  the  theme  of 

the  conferencoi  namely  "Statistical  Design  and  Analysis  of  Experiments."  j 

Titles  of  the  technical  sessions  were:  "Time  Series  and  Stochastic 

Modeling";  "Analysis  of  Variance  Models";  "Statistical  Theory";  "Statistical  | 

Inference";  "Special  Applications";  and  "Material  Reliability".  | 

An  Important  feature  of  these  annual  conferences  Is  the  awarding  of  the 

Samuel  S.  Wilks  Memorial  Medal.  The  1978  award  went  to  the  distinguished  ^ 

scientist  Dr.  William  H.  Kruskal*  Professor  of  Statistics  at  the  University  |! 

of  Chicago.  His  contributions  to  the  field  of  statistics  have  been  truely  >> 

outstanding. 


Professor  Brian  L.  Joiner 
University  of  Wlsconsln-Madtson 

Professor  Richard  E.  Barlow  with 
Bernard  Davis 

University  of  California -Berkeley 


The  Army  Mathtmatlcs  Steering  Committee  (AMSC),  an  Intra-Army  committee, 

sponsors  the  design  conferences  on  behalf  of  the  Chief  of  Research,  I 

Development  and  Acquisition.  Members  of  this  committee  appreciated  the  ! 

fact  that  the  Mathematics  Research  Center  (MRC)  was  willing  to  serve  as  | 

host  for  the  Twenty-Fourth  Conference  on  the  Design  of  Experiments.  They  ij 

would  like  to  thank  Professor  Bernard  Harris  for  serving  as  Chairman  on  i 

Local  Arrangements.  He  was  ably  assisted  In  this  capacity  by  Mrs.  Gladys 

G.  Moran.  Those  In  attendance  appreciated  the  assistance  these  and  other  I 

members  of  NRC  gave  them  with  the  many  problems  that  arose  during  the  | 

course  of  this  meeting.  i 

I 

The  AMSC  has  requested  that  these  Proceedings  be  published  end  distributed  j 

Army-wide  In  order  that  the  Information  contained  therein  will  assist  | 

scientists  with  some  of  their  statistical  problems.  Finally,  committee 


iv 


members  would  like  to  thank  the  Program  Committee  for  all  the  work  It 
did  In  putting  together  another  successful  scientific  conference. 


Gerald  Andersen 
Carl  Bates 
Larry  Crow 
Francis  Dressel 
Walter  Foster 


Bernard  Harris 
Clifford  Maloney 
Douglas  Tang 
Malcolm  Taylor 
Michael  White 


Frank  Grubbs  (Program  Committee  Chairman) 
Robert  Launer  (Conference  Secretary) 

Herbert  Solomon  (Chairman  of  the  Conference) 
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STATISTICAL  ANALYSES  OF  A WEATHER 
MODIFICATION  EXPERIMENT* 


Ralph  A.  Bradley 


Department  of  Statistics 
Florida  State  University 
Tallahassee,  Florida 


I.  INTRODUCTION  AND  SUMMARY 

This  article  is  a sununary  of  a manuscript  [4]  prepared  for 
inclusion  in  a special  issue  on  weather  modification  of  the  ref- 
erenced journal.  The  subject  area  is  one  of  national  importance 
as  emphasized  in  the  recent  reports  [12,13]  of  the  Weather  Modi- 
fication Advisory  Board. 

Phase  I of  the  Santa  Barbara  Convective  Seeding  Test  Program 
was  conducted  by  North  American  Weather  Consultants  (NAWC)  in  the 
Santa  Barbara  area  of  California  from  1967  through  1971.  Details 
of  this  research  were  reported  in  [5,9],  with  the  first  report  us- 
ing augmented  ralngage  data.  Initial  data  analysis  was  reported 
by  NAWC  and  additional  exploratory  analyses  are  summarized  here, 
in  [2,4],  and  in  technical  reports  [1,3,10],  the  second  giving 
additional  detail.  We  are  indebted  to  NAWC  for  their  courtesy 

*A  summary  of  research  supported  by  the  U.S.  Office  of  Naval 
Research  under  Contract  No,  N00014-76-C-0394.  Reproduction  in 
whole  or  in  part  is  permitted  for  any  purpose  of  the  United  States 
Government . 
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in  providing  data  tapes  to  us.  The  Phase  I experimentation  was 
followed  by  Phase  II  research  with  some  experimental  design 
changes.  Only  Phase  I data  are  considered  here. 

The  exploratory  statistical  analyses  reported  are  parametric 
and  Intended  to  lead  to  Insights  that  may  be  checked  with  Phase  II 
data.  There  are  problems  of  validity  of  parametric  assumptions 
and  Issues  of  experimental  design,  multiplicity  of  analyses,  and 
possible  need  for  randomization  analyses  raised  in  [12]. 

We  review  the  Phase  I Santa  Barbara  experiment  and  data 
available.  An  experimental  unit  is  a single  "seedable"  convec- 
tive band  occurring  in  a winter  storm  that  may  have  one  or  more 
such  convective  bands.  The  choice  of  experimental  unit  has  the 
advantage  of  Increasing  the  number  of  available  experimental  units 
in  a season  and  the  possible  disadvantages  of  serial  correlation 
and  persistence  of  seeding  effects  from  unit  to  unit.  Raingage 
data  are  available  for  both  a Target  and  a Control  Area;  concomi- 
tant cloud  physics  data  were  recorded  and,  after  summarization  by 
Gleeson  [10],  used  in  trial  covariance  analyses. 

The  problem  of  data  summarization  is  addressed  first.  It  is 
found  that  use  of  response-surface  methods  is  not  advantageous. 

Use  of  concomitant  variables  for  the  reduction  of  experimental 
errors  in  analyses  reduces  also  the  apparent  effect  of  cloud  seed- 
ing. When  storm  effects  are  included  as  components  of  a paramet- 
ric model,  they  are  found  to  be  totally  or  partially  confounded 
with  seeding  and  again  the  apparent  effect  of  seeding  is  reduced. 
Some  brief  comments  are  included  on  some  multivariate  analyses. 

The  effect  of  our  analyses  is  to  leave  some  doubt  as  to  the  efficacy 
of  cloud  seeding  in  the  Phase  I Santa  Barbara  experiment. 

II.  The  Phase  I Santa  Barbara  Experiment 

The  geographical  setting  of  the  Phase  I Santa  Barbara  Experi- 
ment is  shown  in  Figure  1 . Control  and  Target  or  Test  Areas  were 
designated;  rainfall  was  measured  through  series  of  raingages,  some 
of  which  are  shown  in  the  figure.  While  the  objective  of  the  ex- 
periment was  not  precisely  defined,  it  can  best  be  described  as  an 
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FIG.  1.  Santa  Barbara  Pyrotechnic  Seeding  and  Control  Test  Areas. 
Source:  Figure  2,  Elliott,  St.  Amand,  and  Thompson  [7].  Rain- 
gage  sites  are  designated  by  solid  or  open  circles,  telemetered 
gages  underlined.  The  seeding  and  radar  site  Is  Indicated  by  a 
solid  triangle  and  Santa  Barbara  Airport  by  a solid  square. 


investigation  of  whether  cloud  seeding  can  enhance  precipitation 
within  a rather  large  target  area. 

The  experimental  unit  was  a suitable  convective  band  in  a 
winter  storm  that  might  contain  one  or  more  such  bands  in  a se- 
quence. The  seeding  decision  was  randomized  effectively  and  care 
was  taken  that  the  decision  was  not  known  to  the  meteorological 
analyst  determining  raingage  band  passage  times  and  precipita- 
tions. Criteria  for  determination  of  seedable  convective  bands 
were  specified  but  some  operational  difficulties  were  encountered. 

Band  precipitation  data  were  obtained  from  all  raingages  in 
control  and  target  areas  operational  for  a band.  Air-mass  char- 
acteristics of  each  band  were  determined  from  radiosonde  observa- 
tions at  Santa  Barbara  Airport,  occasionally  replaced  by  Vanden- 
berg  Air  Force  Base  observations.  Gleeson  [10]  defined  and  sum- 
marized data  for  each  band  on  the  following  variables:  - 

Mixing  Ratio,  Xj  - 700mb  Wind  Speed,  Xj  - 700mb  Wind  Direction, 

X^  - Moan  Wind  Speed,  Xg  - Direction  Avg.  Vector  Wind,  Xg  - SOOmb 
Temperature,  X.^  - Stability  Class,  Xg  - Showalter  Index,  Xg  - 
Stability  Wind  Speed,  X^g  - Direction  Stability  Wind,  X^^  - Insta- 
bility Transport,  X^2  " Passage  Time  (Seeding  Site).  The 
data  array  for  the  Phase  I experimentation  may  be  viewed  as  a 
data  matrix  with  N rows  or  bands,  the  first  rows  for  unseeded 
bands  and  the  second  Nj  rows  for  seeded  bands,  N ■ 107,  ■ 51, 

Nj  = 56,  and  with  columns  containing  precipitation  responses  at 
individual  raingages,  possible  grouped  by  locations,  and  values 
of  the  concomitant  variables,  X^  to  Xj2'  The  data  are  not  with- 
out problems.  Raingage  precipitation  responses  are  correlated, 
data  are  missing  for  many  raingages,  rows  may  not  be  independent 
observation  vectors,  and  there  may  be  a persistence  effect  of 
seeding. 

The  main  NAWC  approach  to  data  analysis  was  on  a raingage 
station-by-station  basis.  Single  and  double  ratio  indices  of 
precipitation  were  calculated  and  contour  plots  based  on  these 
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ratios  over  control  and  target  areas  were  given  in  various  reports. 
Let  denote  precipitation  at  station  i for  band  o,  a « 1,..., 

N.  Let  Y (i)  > 1 or  0 as  station  i was  or  was  not  operable  for 
band  a and  let  * 1 or  0 as  band  a was  or  was  not  seeded. 

Then  ■ N(i)  and  respectively  the 

number  of  observations  and  the  number  of  seeded  bands  recorded  at 
station  1.  The  number  of  unseeded  bands  at  station  i is  " 

N(i)-N. (i).  Then 

and 

are  precipitation  averages  at  station  i for  seeded  and  unseeded 
bunds,  Six  control-area  detection  stations  were  used,  stations 
circled  in  Figure  1.  If  k indexes  these  control  stations,  de- 
fine, 

Cg  - |Tg(k)/6 
and 

'ns  ■ 

The  double  ratio  of  NAWC  at  station  1 is 

DRCiD  - [TsCi)/Cg]/[t„gCi)/C„3] 
and  the  single  ratio  is 

SR(i)  - t^(i)/TngCi). 

Much  the  same  contour  plots  were  obtained  from  both  ratios.  Use 
of  the  double  ratio  represents  a use  of  control  area  precipitation 
as  a covarlate. 

The  Wllcoxon-Mann-WIiltney,  two-sample,  rank  test  was  used 
also  by  NAWC  to  assess  the  significances  of  double  and  single 
ratios  for  each  raingage  station.  The  method  of  application  is 
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not  clear  in  reports  but  is  understood  as  follows.  For  the  single 
ratio,  was  used;  the  precipitation  themselves  were  grouped 

into  two  samples,  seeded  and  unseeded,  and  the  rank. tost  applied. 
For  the  double  ratio,  was  calculated  for  each  band  a at 

station  i,  being  the  average  of  the  six  control  area  detec- 
tion stations  for  band  o,  and  these  indices  were  grouped  into  two 
samples  as  before. 

NAWC  was  aware  tliat  these  significance  tests  were  open  to 
possible  criticisms,  particularly  dependencies  from  station  to 
station.  A limited  Monto  Carlo  study  was  conducted  and  reported 
by  Elliott  and  Brown  [6].  They  state:  "At  the  0,05  significance 
level  for  all  bands,  29  stations  in  the  original  test  sample  were 
found  to  show  a positive  difference  between  seeded  and  not- seeded 
cases  (bands) ; and  three  Monte  Carlo  runs  (out  of  50)  were  found 
to  have  as  high  or  higher  counts  of  stations  with  a positive  dif- 
ference at  this  significance  level." 

III.  DATA  SUMMARIZATION 

A more  direct  approach  to  analysis  of  the  experiment  is 
through  summary  measures  of  precipitation  for  each  experimental 
unit  over  designated  response  areas. 

Bradley,  Srivastava  and  Lanzdorf  [1,2]  defined  response  areas 
as  in  Table  I that  may  be  located  in  Figure  1.  The  numbers  of 
raingage  stations  and  the  data  used  for  Target  Areas  (l)-(iv)  are 
those  of  the  Bureau  of  Reclamation  study  [5]  and  those  for  Target 
Area  (v)  are  those  of  the  Naval  Weapons  Center  study  [9]  with 
minor  modifications  noted  in  [3,4]. 
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*The  Control  Area  for  the  Naval  Weapons  Center  study  con- 
sists of  all  39  stations  West  of  the  seeding  site. 

Use  of  a simple  average  over  stations  is  the  most  direct 
method  of  data  summarization  for  a convective  band.  Overall  pre- 
cipitation means  are  reported  in  Table  II.  Note  that  the  Control 
Area  mean  is  higher  for  seeded  bands  suggesting  that  seeding  may 
have  had  some  effect  in  the  Control  Area  or  misfortvine  in  the  ran 
domized  choice  of  bands  to  be  seeded. 

TABLE  II 

Precipitation  Means  in  Inches 


(ill) 


(Iv) 

(V) 

Control  1 

0.271 

0 267 

0.234 

0.187 

ti.l90 

0.203 

Response  Areas 


Seeded  Bands 
Unseeded  Bands 


With  a view  to  improved  data  suimnarizatlon,  Brr  ley,  Srivas- 
tava  and  Lanzdorf  [1,2]  fitted  response  surfaces  sei  rately  for 
the  Control  Area  and  Target  Area  (1)  with  the  coord^  ^ates  of  lati- 
tude and  longitude  of  ralnguge  stations  as  independ  t variables 
and  ralngage  precipitation  as  the  dependent  variabl''  . General 


two-dimensional  cubic  response  models  were  necessary  to  represent 
responses  adequately.  Separate  response  surfaces  were  found  for 
each  convective  band.  Precipitation  volumes  and  their  variances 
were  calculated  over  the  designated  target  areas  and  Control  Area. 
Figure  2 is  typical  of  results  obtained;  the  region  where  the  sur- 
faces is  negative  is  off-shore. 

The  response  surface  approach  was  successful  as  a method  of 
data  summarization  in  that  some  70%  of  the  Inherent  variation  in 
responses  among  ralngages  within  a band  and  response  area  was  ex- 
plained by  the  independent  variables.  It  was  not  successful  in 
Improvement  of  data  summarization  in  comparison  with  use  of  the 
means  of  ralngages  within  response  areas  for  a convective  band 
in  that  correlations  between  precipitation  volumes  calculated  from 
the  response  surfaces  and  precipitation  means  ranged  from  0,97  to 
0,99  for  Target  Areas  (l)-Civ)  and  the  correlation  was  0.89  for  the 
Control  Area.  Thus,  the  use  of  volume.s  cannot  be  expected  to 
yield  now  insights. 

Scott  [10]  used  a multivariate  approach  to  data  summarization. 
He  found,  with  some  difficulty  and  innovation,  principal  components 
among  raingage  responses  in  both  Target  Area  (i)  and  the  Control 
Area.  The  first  three  principal  components  were  interpretable 
approximately  as  a mean  response,  a coastal  versus  inland  con- 
trast, and  an  East-West  contrast.  Percentages  of  variation  ex- 
plained by  these  components  were  respectively  71.3,  6.7  and  5.9 
in  Target  Area  (i)  and  76.1,  6.7  and  4.7  in  the  Control  Area.  The 
correlations  of  the  first  component  with  the  band  mean  were  0.997 
for  Target  Area  (i)  and  0.985  for  the  Control  Area.  Scott  is  en- 
gaged in  the  use  of  these  results  in  examination  of  the  effects 
of  seeding;  it  seems  unlikely  that  much  additional  Information  will 
be  forthcoming. 

In  the  following  section,  we  show  some  parametric  analyses  for 
Target  Area  (i).  Although  Bradley,  Srivastava  and  Lanzdorf  [3] 
followed  through  with  analyses  on  precipitation  volumes  as  well 
as  means,  we  report  only  on  the  use  of  means.  All  Target  Areas 
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FIG.  2.  Graph  of  Cubic  Response  Surface:  Band  96  (Seeded),  Tar 
get  Area  (i).  Source:  Figure  2,  Bradley,  Srlvastava,  and  Lanz- 
dorf  [1].  Vertical  axis  is  2.3  times  precipitation  in  inches. 
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of  Table  I gave  similar  results. 

IV.  SOME  PARAMETRIC  ANALYSES 

Weather  modification  experiments  are  conducted  necessarily  in 
a natural  environment  Involving  much  variability.  The  use  of  co- 
varlates  in  analyses,  au  summarized  by  Gleeson  [10],  for  the  reduc- 
tion of  experimental  error  appeared  to  be  the  best  means  to  im- 
proved experimental  design. 

Initial  covariance  analyses  were  reported  in  [3] . (Some  la- 
ter analyses  are  summarized  below).  Regression  models  used  were 
of  the  form. 

P 

^ + E + 6Z  + e, 

° i-1  ^ ^ 


where  U is  a precipitation  response  variable  for  a target  area, 
is  the  i-th  covariate,  2 ■ 1 or  0 as  the  convective  band  was 
or  was  not  seeded,  the  B's  and  6 are  regression  parameters,  and 
e is  a random  error.  The  data  matrix  has  rows,  (U^,  Vj^^,..., 

Vp^,  Z^) , a ■ 1,...,N.  The  regression  parameters  were  estimated 

by  weighted  least  squares  through  minimization  of 


“a"'. 


-So-! 

^ i-1 


B,V, 
i lo 


- 62  )‘ 
a 


In  [3] , use  of  the  listed  set  of  covariates  and  their  Interactions 
with  seeding,  along  with  X^,  a measure  of  Control  Area  precipita- 
tion, was  explored.  In  analysis  of  variance  tables  in  [3]  and 
below,  sources  of  variation,  when  included  in  models,  were  ordered; 
covariates,  covariate  by  seeding  Interactions  after  adjustment 
for  covariates,  and  seeding  after  adjustment  for  covariates  and 
interactions. 
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The  use  of  covariates  was  effective  in  reducing  experimental 
error  but  less  so  was  the  use  of  covariate  by  seeding  interactions. 
Preliminary  analyses  and  redundancy  considerations  suggested  re- 
duction from  12  covariates  to  7:  X2«  X^,  Xg,  X^,  Xg,  X^^  and 

Xj2«  along  with  X^,  Control  Area  mean  precipitation.  There  is 
concern  about  use  of  X^  because  Elliott  and  Thompson  [8]  suggest 
the  possibility  of  an  up-wind  effect  of  seeding  west  of  the  seed- 
ing site  attributable  not  to  westward  seeding  contamination  bit 
to  a seeding-caused  blocking  of  the  air-mass  flow  leading  to  up- 
wind convection  development.  This  may  have  affected  X^j*  band 
passage  time  at  the  seeding  site,  also.  Two  choices  of  weighting 
were  used,  w ■!  (unweighted]  and  w ••  n /s  when  U was  target 

01  Qt  tt  Ot  ^ 

area  mean  precipitation,  where  n was  the  number  of  observations 

a 2 

contributing  to  the  precipitation  mean  for  band  o and  s was  the 

variance  among  those  observations.  Weighted  analyses  with  w ■ 

2 “ 
n^/s^  were  less  satisfactory  than  unweighted  analyses.  This  dif- 
ficulty arose  because  standard  deviations  are  proportional  to  meane 
and  very  high  weights  were  associated  with  convective  bands  with 
low  precipitations.  The  two  best  covariates  were  X^,  and  Xjj* 
The  use  of  covariates  reduced  the  apparent  effect  of  seeding.  In 
these  analyses  and  those  below,  N ■ 106  when  covariates  are  used 
because  covariate  data  were  missing  for  Band  73. 

Analyses  were  redone  in  [4]  with  responses  transformed  log- 
arithmically to  stabilize  variances.  The  transformed  variate  z 
has  the  form,  log  (l4'ay],  where  Y is  a target  area  precipitation 
observation.  For  Target  Area  (i) , U is  now  the  target  area. mean 
of  z,  and  w^  = n^.  The  regression  analyses  are  similar  to  those 
described  above.  Models  with  and  without  and  Xj2  were 

used  because  it  has  been  suggested  that  they  may  have  been  affec- 
ted by  seeding.  .Six  models  were  used  as  follows: 


n 


Model 

Identification  of  Vj,...,Vp 

(1) 

No  covarlates. 

(2] 

^C*  ^2*  ^3'  ^7'  ^8'  ^11' 

(3] 

Model  (2)  less  X^. 

(4] 

Model  (2)  less  X^2* 

C5] 

Model  (2)  less  X^,  Xj^2‘ 

(6] 

Model  (2)  plus  XjZ,  X^Z,  XgZ, 

X,Z.  X3Z,  Xj^Z,  Xj2Z. 

2 

Mean  squares,  values  of  P,  and  values  of  R , the  coefficient  of 

determination,  for  these  models  are  shown  in  Table  III. 

We  comment  on  the  analyses  of  Table  HI.  The  transformation 

was  shown  to  be  effective  in  stabilizing  variances  except  for 

small  values  of  U,  values  for  convective  bands  that  may  not  have 

been  acceptable  "seedable"  bands.  Examination  of  residuals  about 

regression  models  for  the  transformed  data  suggests  that  transform 

mation  improved  symmetry  and  approximate  normality  of  their  dis- 

2 

tributlons.  Slightly  larger  values  of  R were  obtained  from  the 

2 

transformed  data  than  in  [3].  Results  for  model  (5)  show  that  R 
is  reduced  considerably  when  and  omitted  as  covarl- 

ates.  Results  for  model  C6)  show  that  interaction  terms  contribute 
little. 

There  is  little  indication  for  models  (2) -(6)  in  Table  III 
of  any  effect  of  seeding.  For  Model  (1),  the  one-sldod  signifi- 
cance level  is  0.06,  consistent  with  the  randomization  analysis 
reported  in  Section  II.  The  covariates  may  be  affected  by  seeding. 
We  have  commented  on  this  in  regard  to  X^,  and  Xj^*  Gleeson  [10] 
saw  small  but  consistent  differences  in  covarlates  for  seeded  and 
unseeded  bands.  The  otlier  covariates  were  based  on  radiosonde 
data  taken  at  Santa  Barbara  Airport,  well  into  the  target  area. 
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TABLE  III 

Analyses  of  Variance,  Transformed  Data, 
Target  Area  (i)*  Models  (l)-(6) 


Model 


Source  of 
Variation 


Seeding 

Interactions 

Covariates 

Error 


1 (» ■ 

C2) 

C3) 

1 d.f.  M.S. 

■ 

d.f.  M.S. 

■ 

d.f.  M.S.  F 

1 110.3  2.77  1 0.6  .05  1 0.0  .00 

- - 8 382.5  30.79  7 324.4  22.24 

104  39.8  - 96  12.4  - 97  16.8  > 

- 0.03  - - 0.72  - - 0.62 


Source  of 
Variation 


Seeding 

Interactions 

Covariates 

Error 


d.f.  M.S. 


1 11.6  .78 

7 398.6  26.64 
97  15.0  - 

0.66 


(5) 

(6) 

d.f.  M.S.  F 

d.f.  M.S.  F 

1 44.7  1.46 

1 1.6  .13 

- 

7 17.0  1.41 

6 200.4  6.53 

8 382.5  31.73 

98  30.7  - 

1 

89  12.1  - 

- 0.29 

- - 0.75 

Some  other  sources  of  variation  have  not  been  considered  yet. 
Convective  bands  occur  within  ''inter  storms  with  one  or  more  bands 
Therefore  there  is  a total  or  partial  confounding  of  storm  ef- 
fects with  seeding.  If  covariates  are  omitted  and  the  effect  of 
seeding  is  considered  after  adjustments  for  storms,  we  have  the 
analysis  of  Table  IV.  It  is  seen  that  R ■ 0.5U,  con^jarable  to 


values  of  in  Table  III;  more  degrees  of  freedom  are  expended. 


But  the  apparent  effect  of  seeding  has  disappeared  again.  In 
future  similar  experimentation,  use  of  storms  for  blocking 
should  be  considered,  perhaps  as  suggested  in  [12],  with  random- 
ization within  storms  rather  than  over  all  convective  bands  as 
done  in  the  Phase  I Santa  Barbara  experiment  for  which  the  analy- 
ses reported  in  Section  II  and  in  Table  III  for  model  (1)  seem 
appropriate. 

In  further  exploratory  analyses,  we  considered  as  additional 
sources  of  variation  position  of  the  band  within  a storm  and  a 
possible  first-order  carry-over  effect  of  seeding  from  a seeded 
band  to  the  following  band  if  in  the  same  storm.  No  real  effects 
for  position  or  carry-over  were  found. 

The  analyses  of  this  section  are  open  to  technical  concerns, 
but  parametric  methods  provide  the  best  means  of  exploratory  anal- 
ysis if  not  for  the  exact  determination  of  significance  levels. 

The  possible  persistence  effect  of  seeding  raises  questions  about 
the  independence  of  experimental  units  that  may  be  subject  also  to 
serial  correlaton.  Normality  assumptions  are  not  valid  for  indi- 
vidual raingage  observations  but  may  be  appropriate  for  target  area 


means.  Some  variance  heterogeneity  is  present  after  transforma- 
tion of  the  data.  Choice  of  weights,  » n^,  for  analyses 
with  transformed  data  is  only  strictly  appropriate  if  raingage 
observations  are  independent.  Covarlates  in  regression  models 
are  subject  to  experimental  errors. 

V.  REMARKS 

Some  remarks  and  recommendations  can  be  made  after  analysis 
of  the  Phase  t Santa  Barbara  experiment.  We  are  in  near  agree- 
ment with  the  conclusion  of  Elliott  and  Brown  [6]:  "Even  when 
those  bands  not  as  receptive  to  seeding  were  included  In  the 
sample,  the  seeded  to  not-seeded  precipitation  Increases  wore 
greater  than  S0%.*'  The  means  of  Table  11  show  Increases  near 
to  50%  and  the  analysis  of  Table  III  for  Model  (1)  suggests  sig- 
nificance near  to  the  0.05  level. 

Improved  experimental  design  is  needed  but  not  easy  to 
achieve.  Use  of  convective  bands  as  experimental  units  increases 
the  number  of  available  units  per  season  but  raises  other  prob- 
lems. Some  In^rovements  are  needed: 

(1)  Improved  detection  and  determination  of  "seedable" 
bands. 

(ii)  More  uniform  dispersement  of  ralngages  over  regions 
of  Interest. 

(ill)  Improved  determination  and  measurement  <;f  precipita- 
tions attributed  to  particular  convective  bands. 

(iv)  Better  determination  and  measurement  of  covariates 
free  from  possible  seeding  effects. 

(v)  Allowance  for  blocking  by  storms  for  fii  -ther  control 
of  variation.  Concerns  may  remain  in  regard  to  dep  ndencies 
among  experimental  units,  data  transformation,  var  ice  hetero- 
geneity and  persistence  effects  of  seeding,  some  o^  /hich  may  be 
met  through  use  of  randomization  analyses.  Puther  loteorological 
and  statistical  research  is  needed. 
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CANDIDATE  SEQUENTIAL  DESIGNS  FOR 
OPTIMUM  SEEKING 

Carl  B.  Bates 

US  Army  Concepts  Analysis  Agency 
Bethesda,  Maryland 


ABSTRACT.  Phase  I of  the  Target  Acquisition  Systems  Force  Mix 
Evaluation  Analysis  (TASFMA)  Study  Is  the  development  of  a methodology 
for  evaluating  the  combat  effectiveness  of  mixes  of  systems  within  a 
functional  area.  The  methodology  requires  a division  combat  simulation 
model  whose  outcome  Is  sensitive  to  changes  In  the  quality  and  quantity 
of  sensor  systems  deployed.  DIVOPS,  a division-level  combined  arms  en- 
gagement model,  Is  tne  primary  candidate  for  the  combat  simulation 
model.  The  model  will  simulate  multiple  quantities  of  up  to  15  differ- 
ent sensor  types.  Because  the  DIVOPS  Model  can  represent  15  sensor  sys- 
tems, the  developed  methodology  must  be  capable  of  accommodating  15  sys- 
tem types.  Additionally,  It  Is  desired  that  three  Quantities  be  exa- 
mined for  each  of  the  16  sensor  types.  Although  DIVOPS  Is  a relatively 
fast  running  model,  all  3^®  runs  are  Impossible.  Consequently,  a method 
Is  needed  for  constructing  a manageably  sized  subset  of  the  total 
Input  combinations.  This  paper  presents  candidate  sequential  designs 
for  the  study  and  search  of  the  optimum  sensor  mix.  Two  two-level  de- 
signs are  presented,  a resolution  III  design  which  requires  16  runs  and 
a resolution  V design  which  requires  256  runs.  Then  two  three-level  de- 
signs are  presented,  a "Minimum  Number  of  Points"  design  requiring  136 
runs  and  a fractional  factorial  design  requiring  243  runs.  Advant- 
ages and  disadvantages  of  the  designs  are  discussed. 

1.  INTRODUCTION.  The  Target  Acquisition  Systems  Force  Mix  Evalua- 
tion Analysis  (TASFMA)  Study  consists  of  two  phases.  Phase  I Is  the  de- 
velopment of  a methodology  for  evaluating  the  combat  effectiveness  of 
mixes  of  systems  within  a functional  area.  Phase  II  Is  the  demonstra- 
tion of  the  usefulness  of  the  methodology.  The  following  sequential  de- 
signs are  proposed  for  Incorporation  Into  the  developed  methodology. 

The  methodology  requires  a division  combat  simulation  whose  combat 
outcome  Is  sensitive  to  changes  In  the  quality  and  quantity  of  sensor 
systems  deployed.  DIVOPS  Is  the  primary  candidate  for  the  combat  simu- 
lation. DIVOPS  is  a two-sided,  deterministic,  division-level  ground 
combat  model.  The  model  will  simulate  multiple  quantities  of  up  to  15 
different  sensor  types.  The  model  documentation  Is  In  Reference  1. 
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2.  PROBLEM  DESCRIPTION  AND  BACKGROUND.  Because  the  DIVOPS  Model 
can  represent  16  sensor  types,  the  developed  methodology  must  also  be 
capable  of  accommodating  lb  system  types.  Additionally,  because  sensor 
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influence  may  be  nonlinear,  it  is  essential  that  more  than  just  two 
quantities  (say,  a low  and  a high  number)  of  sensors  be  examined  for 
each  of  the  different  types.  It  is  desired  that  three  quantities  be 
examined  for  each  of  the  15  sensoc  types.  Although  DIVOPS  is  a rela- 
tively fast  running  model,  all  3^®  input  combinations  cannot  be  run 
since  14,000,000.  Moreover,  neither  could  thertnodel  be  exercised 

for  all  2'^°  input  combinations.  If  desired,  since  ■ 32,768.  Conse- 
quently, a method  is  needed  for  constructing  a manageable  size  subset  of 
the  totality  of  the  3^“  input  combinations,  hereafter  referred  to  as  de- 
sign points.  The  purpose  of  this  paper  is  to  present  proposed  sequen- 
tial designs  for  the  above  described  optimization  problem. 


3,  METHODOLOGY  RATIONALE.  The  methodology  presupposes  little  or 
no  a priori  information  about  the  functional  relationship  between  model 
inpuit  variables  and  model  output  variables.  The  model  input  variables 
and  output  variables  are  considered  to  be  continuous  variables.  For  the 
purpose  of  this  paper,  it  is  assumed  that  the  three  quantities,  here- 
after termed  levels,  of  sensors  or  systems  are  known  for  each  system 
under  investigation.  That  is,  the  proposed  procedure  is  not  for  deter- 
mininy  what  the  levels  should  be.  Rather,  it  is  for  identifying  the 
combinations  of  existing  levels  which  should  be  employed  in  the  search 
for  the  optTmum  systems  mix. 


Although  interest  is  in  three  levels  of  each  of  the  input  vari- 
ables, because  of  the  magnitude  of  it  is  felt  that  the  examination 
process  must  be  sequential  in  nature,  That  is,  the  search  should  start 
with  a 2*^  design  and  proceed  to  a 3^  design  where  k'  i k.  This  ap- 
proach employs  some  of  the  screening  concepts  of  experimental  designs 
and  response  surface  fitting. 


4.  TWO-LEVEL  DESIGNS 


a.  General.  Let  x. ,1  ■ 1,2 k be  the  model  input  variables, 

where  k ■ 15,  Denote  the  extreme  two  of  the  three  levels  of  each  of  the 
variables  by  "0"  and  "1,"  respectively.  If  k were  three,  the  eight  de- 
sign points  would  be  as  shown  in  Table  1 and  are  geometrically  illus- 
trated in  Figure  1. 


If  the  full  factorial  experiment  were  performed  and  a dependent 
variable  y were  measured  or  observed  at  each  of  the  eight  independent 
variable  combinations,  the  full  model  shown  as  Equation  [1]  could  be 
fitted. 


^ “ ^000  ^ ^100^  * ^010’‘2  *^001^3  ‘’110^1^2 


’’101^1^3  ^OllVa  ^ 


[1] 
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The  general  full  2*^  design  permits  the  fitting  of  the  following  2*^"^ 
terms  plus  the  intercept,  Pq. 


k 

k(k-l)/2 

k(k-l)(k-2)/2x3 


single-variable  terms 
two-variable  terms 
three-variable  terms 


Because  a full  215  experiment  Is  Impossible  for  the  TASFMA  optim- 
ization problem,  two  alternative  (1/2)P  fractional  designs,  2'^"P,  are 
presented.  The  first  design  permits  fitting  the  15  single-variable 
terms  only.  The  second  design  permits  fitting  the  15  single-variable 
terms  and  the  (15xl4)/2  - 105  two-variable  terms. 

b.  Resolution  III  Design.  Resolution  III  designs  are  available 
which  require  k+1  runs  to  study  k variables,  where  k+1  is  a multiple  of 
four.  In  Reference  2,  Box  and  Hunter  give  the  following  definition  of 
resolution  III  designs: 


"No  main  effect  Is  confounded  with  any  other  main  effect,  but 
main  effects  are  confounded  with  two-factor  Interactions  and 
two-factor  Interactions  with  one  another." 


The  design  is  first  illustrated  for  a seven-variable  experiment. 
Consider  a (1/2)^  of  a complete  2'  factorial,  a 2'"^  design.  Construc- 
tion of  the  design  matrix  starts  with  the  design  matrix  In  Table  1,  a 
full  2'^.  Four  additional  columns  are  generated  from  the  three  original 
columns.  Treat  a "0"  as  a "-1"  and  a "1"  as  a "+1"  and  product  the 
three  pairs  of  columns  and  the  one  triple.  That  1s,  generate  columns  4, 
5,  6,  and  7 as  follows: 


column  4 » column  1 times  column  2 

column  5 ■ column  1 times  column  3 

column  6 ■ column  2 times  column  3 

column  7 ■ column  1 times  column  2 times  column  3 
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The  resulting  design  matrix  Is  shown  In  Table  2 and  permits  fitting 
the  seven-d1mens1onal  plane  given  In  Equation  [2]. 


Table  2,  Design  Matrix 


’‘I 

X2 

X3 

H 

X5 

>^6 

X7 

0 

0 

0 

1 

1 

1 

0 

1 

0 

0 

0 

0 

1 

1 

0 

1 

0 

0 

1 

0 

1 

0 

0 

1 

1 

0 

0 

1 

1 

1 

0 

1 

0 

0 

0 

1 

0 

1 

0 

1 

0 

0 

0 

1 

1 

0 

0 

1 

0 

1 

1 

1 

1 

1 

1 

1 

y 

" “o 

+ bj^Xi  + 

•>2*2 

+ 

^4^4 

+ bg) 

tg  + b 

+ b^x 

Applying  the  technique  described  In  the  previous  paragraph  to  the 
15  variables  of  the  TASFMA  Study  gives  the  design  matrix  In  Table  3, 

The  first  four  columns  constitute  the  design  for  a full  ?.  factorial. 
Columns  5 through  10  are  pairwise  products  of  the  first  four  columns. 
Columns  11  through  14  are  the  four  triple  products,  and  column  15  1s  the 
product  of  all  four  columns.  The  products  are  Indicated  under  the  col- 
umn heading  In  the  table.  Exercising  the  computer  simulation  model  for 
each  of  the  16  Input  variable  combinations  Indentifled  In  Table  3 would 
permit  fitting  a 15-d1mens1onal  plane  for  each  output  variable  under 
study.  For  each  output  variable,  this  would  give  a function  in  terms  of 
each  of  the  15  Input  variables.  Equation  [3]. 


y ■ bQ  + b^xi  + bgXg  + ...  + bi4Xi4  + b^gx^g 


[3] 


The  regression  coefficients  b^,  ...,  b^5  are  the  slopes  of  the 

plane  In  the  respective  dimensions.  The  slopes  can  be  analyzed  to  as- 
sess the  effect  each  of  the  15  Input  variables  has  upon  the  particular 
output  variable  and  also  to  refine  the  space  which  will  be  further 
examined  with  a three-level  design. 
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The  assessment  will  give  the  direction  of  the  maximum  response  as  well 
as  an  ordering  of  the  relative  contribution  of  each  of  the  15  sensor 
types.  If  some  of  the  15  contribute  very  little  as  compared  to  the 
others,  then  the  low  contributors  can  be  fixed  at  either  their  upper  or 
constraining  levels  for  the  follow-on  three-level  examination. 

The  above  "cheap"  experiment  is  not  without  risk.  Naturally,  a 
sacrifice  has  to  be  made  for  a design  with  such  a small  number  (16)  of 
design  points.  If  two  input  variables,  Xi  and  x^,  significantly  inter- 
act and  neither  Xj  nor  X4  individually  significantly  contribute,  the 
variables  would  not  be  recognized  as  being  sufficiently  important  for 
inclusion  in  the  follow-on  three-level  Investigation.  If  this  risk  is 
considered  too  great  and  a design  is  required  which  will  permit  the 
testing  of  all  of  the  two-variable  interactions,  then  the  price  of  many 
more  computer  simulation  model  runs  must  be  made.  In  any  case,  the 
resolution  III  design  should  be  conducted  first  because  it  requires  so 
few  runs. 


c.  Resolution  V Design.  Resolution  V designs  are  defined  by  Box 
and  Hunter  in  Reference  2 eis: 


"No  main  effect  or  two-factor  Interaction  is  confounded  with 
any  other  main  effect  or  two-factor  interaction,  but  two- 
factor  Interactions  are  confounded  with  three-factor  interac- 
tions." 


Box  and  Hunter  discuss  resolution  V designs  and  their  construction 
in  Reference  3,  This  is  the  type  of  fractional  factorial  designs 
conventionally  introduced  in  experimental  design  texts,  such  as.  Refer- 
ences 4,  5,  6,  7,  8,  and  9.  The  construction  of  resolution  V de- 
signs is  not  the  purpose  of  this  paper  and  will  not  be  discussed  here. 
Their  construction  can  be  found  in  the  above  references. 

A IB-variable  experiment  has  k ■ 15  main  effects  and  k(k-l)/2  ■ 105 
two-variable  interaction  effects.  The  smallest  resolution  V design  for 
a experiment  has  256  design  points.  Using  the  15  letters  A,  B 

P (excluding  I)  to  represent  the  15  input  variables  Xj,X2 ^3^5  and 

the  identifying  contrast  shown  in  Table  4,  gives  the 

In  Table  5.  From  the  256-run  simulation  model  experiment  we  can  fit  the 
model  in  Equation  [4]. 


y ■ Bq  + bjXj  + b2X2  + ...  + b35Xi5  + 
^14,15’‘14’‘15 


[4] 
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Equation  [4]  has  the  15  single-variable  terms  that  are  in  equation 
[3]  plus  105  two-variable  terms.  That  Is,  120  degrees  of  freedom  of  the 
total  255  degrees  of  freedom  are  used  for  fitting.  The  remaining  135 
can  be  used  for  significance  testing.  Testing  significance  of  the  re- 
gression coefficients  can  be  accomplished  as  Illustrated  In  the  ANOVA 
table,  Table  6.  The  testing  will  Identify  those  coefficents  which  are 
not  significantly  different  from  zero.  The  corresponding  variables  can 
then  be  fixed  In  the  manner  discussed  In  paragraph  b above,  and  the  In- 
vestigation can  proceed  to  the  three-level  follow-on  Investigation. 

I 

I 

1 


Table  6.  ANOVA  for  2^^“^  Design 


Source 

DF 

SS 

MS 

F-ratIo 

^1 

1 

SS(bi) 

MS(bi) 

MS(bi)/MS(LOF) 

• 

1 

• 

SS(b2) 

« 

MS(b2) 

a 

MS(b2)/MS(L0F) 

a 

« 

bl5 

• 

1 

• 

SS(bi5) 

a 

MSb(i5) 

a 

MS(bi5)/MS(L0F) 

'’i.z 

• 

1 

« 

SS(bi^2) 

• 

MS(bi,2) 

a 

MS(bi^2)/MS(L0F) 

a 

« 

.^14,15 

• 

1 

• 

a 

a 

MS(bi4j5)/MS(L0F) 

Lack  of  Fit 

135 

SS(LOF) 

MS(LOF) 

Total 

255 

SS (Total) 
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The  obvious  disadvantage  of  the  above  design  is  the  large  number  of 
simulation  model  runs  required.  This  cannot  be  avoided  when  so  many 
(120)  coefficients  are  to  be  estimated  and  tested.  Care  must  also  be 
taken  when  assessing  interaction  significance.  Since  the  interaction  of 
each  variable  with  each  of  the  other  14  variables  is  being  tested,  an 
alpha-percent  of  the  interactions  would  be  expected  to  be  statistically 
significant  due  to  random  chance. 

5.  THREE-LEVEL  DESIGNS 


a.  General . As  with  the  two-level  designs,  two  candidate  three- 
level  designs  are  proposed.  One  design  may  be  applied  to  all  15  input 
variables  if  necessaryi  the  other  design  presupposes  that  the  previous 
screening  process  reduces  the  number  of  input  variables  so  that  k'  < 10. 

First,  the  notation  is  changed  from  that  used  in  the  previous  sec- 
tion. Now,  denote  the  three  levels— low,  middle,  and  high— by  “0",  “1", 
and  "2",  respectively.  If  k*  were  three,  the  27  design  points  of  a full 
S'®  experiment  would  be  as  given  in  Table  7,  The  full  design  is  illus- 
trated geometrically  in  Figure  2. 

If  the  full  design  were  executed,  the  following  model  of  Equation 
[5]  could  be  fitted.  Equation  [5]  has  27  terms. 


y ■ ^000  ^lOO’^l  ■**  *’010^2  ^001^3 

^200’'!  **020^?  **002*3 

**110*1*2  + **101*1*3  **011*2*3 

*’l20*l*i  + •••  + **222*1*2*S 

The  general  full  3*^  design  permits  fitting  3*^  terms,  including  th? 
Intercept.  In  practice,  however,  even  for  moderate  k',  all  possible  3* 
terms  are  seldom  fitted.  Usually  they  are  not  even  desired.  Two  candi- 
date thicee-level  designs  are  proposed  which  require  considerably  less 
than  3*  design  points.  The  first  is  the  "Minimum  Number  of  Points"  de- 
sign. The  second  design  is  the  conventional  3*  "P  fractional  factorial 
design. 
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Table  7.  3^  Design  Matrix 


!L- 

X2 

J2. 

0 

0 

0 

1 

0 

0 

2 

0 

0 

0 

1 

0 

1 

1 

0 

2 

1 

0 

0 

2 

0 

1 

2 

0 

2 

2 

0 

0 

0 

1 

1 

0 

1 

2 

0 

1 

0 

1 

1 

1 

1 

1 

2 

1 

1 

0 

2 

1 

1 

2 

1 

2 

2 

1 

0 

0 

2 

1 

0 

2 

2 

0 

2 

0 

1 

2 

1 

1 

2 

2 

1 

2 

0 

2 

2 

1 

2 

2 

2 

2 

2 

32 


b.  "Minimum  Number  of  Points*'  Design.*  The  "Minimum  Number  of 
Points  (MNOP)"  design  contains  the  same  number  of  design  points  as  there 
are  terms  In  the  fitted  function,  Including  the  intercept.  The  design 
permits  fitting  all  first  and  second  order  terms. 

First,  the  design  Is  Illustrated  for  k'  ■ 3.  The  design  matrix  for 
k'  ■ 3 Is  shown  In  Table  8.  The  ten  design  points  are  Illustrated  In 
Figure  3 and  will  permit  fitting  Equation  [6j. 


Table  8.  Minimum  Number  of  Points  Design 
Matrix  for  k'  -3 


X2 

^2 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

2 

0 

0 

0 

2 

0 

0 

0 

2 

1 

1 

0 

1 

0 

1 

0 

1 

1 

y ■ l>ooo  + biQoXj  + boiQXg  + >>001x3 

^110X1X2  + bl01>‘l’^3  + C®] 

To  show  the  pattern  more  clearly  before  presenting  the  MNOP  design 
for  k'  ■ 15,  the  design  Is  also  Illustrated  for  five  variables.  Table  9 
contains  the  design  for  k'  ■ 5.  Note  that  k'  rows  have  a single  "1",  k' 
rows  have  a single  "2",  and  k'(k'-l)/2  row>  have  two  "I's".  Although 
the  model  that  can  be  fitted  Is  obvious.  It  Is  given  In  Equation  [7]  for 
completeness.  Subscript  notation  Is  changed  from  that  previously  used 
to  conserve  space. 


♦The  design  was  brought  to  the  attention  of  the  author  by  Dr.  George 
Box-'in  a personal  communication  during  May  1978. 
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Table  9.  MNOP  Design  Matrix  for  k'  ■ 5 


X2 

^3 

X4 

^5 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

0 

0 
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0 

1 

0 

0 

0 

0 

0 

1 

0 

0 

0 

u 

0 

1 

2 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0 

0 

2 

1 

1 

0 

0 

0 

1 

0 

1 

0 

0 

1 

0 

0 

1 

0 

1 

0 

0 

0 

1 

0 

1 

1 

0 

0 

0 

1 

0 

1 

0 

0 

1 

0 

0 

1 

0 

0 

1 

1 

0 

0 

0 

1 

0 

1 

0 

0 

0 

1 

1 

y “ bo  + b^xi  + bgxg  + b3X3  + b4X4  + bgXg 
+ biixf  + bggx^  + b33x5 
+ ^44x5  + b55X§  + bi2XiX2 

+ bi3XiX3  + bi4XiX4  + bigXiXg 
+ ^23x2x3  + b24X2X4  + b25X2X5 

I 

■'■  + ‘>35X3><5 

+ i^abj^a^b  [7] 

I 

] 

'} 

'3 


Construction  of  the  MNOP  design  for  15  variables  should  be  obvious 
from  the  above  illustrations.  The  design  has  15  rows  with  a single  "1", 
15  rows  with  a single  "2",  and  105  rows  with  two  "I's",  and  one  row  of 
all  "O's",  giving  a total  of  136  design  points.  The  design  matrix  is 
shown  in  Table  10.  The  columns  denote  the  15  variables  X][  through  xic. 
If  a computer  model  simulation  experiment  were  conducted  using  the  MNOP 
design,  the  model  given  in  equation  [8]  could  be  fitted  for  each  depen- 
dent variable  under  investigation.  Each  fitted  function  could  then  be 
studied  to  determine  the  optimum  systems  mix  for  each  measure  of  effec- 
tiveness. 


y - bg  + bj^Xj  + b2X2  + ...  + bj5Xj5 
+ + b2z4  + ...  + bis^igxjs 

* '*12^1X2  + ‘>13>^1^3  + •••  + bi4j5Xi4Xi5  [8] 


Ifi  however,  the  MNOP  design  is  considered  inadequate  and  the  prior 
two-1eve1  examination  has  resulted  la  a screening  of  the  original  Id  in- 
put variables  down  to  not  more  than  10  variables,  a fractional  factorial 
design  can  be  applied. 

Fractional  Factorial.  If  k'  i 10,  a fractional  factorial 
experiment  can  des'lgned  which  has  243  design  points.  Theoretical 
background  on  3*^  "P  fractional  factorial  designs  can  be  found  In  Refer- 
ences 4,  5,  6,  7,  8,  and  9. 

The  dfislgn  Is  Illustrated  for  k'  ■ 10.  The  design  is  a 1/3®  x 3^^, 
i.e. , a fractional  factorial.  As  in  paragraph  4c  above,  the  ten 
letters  A,  B,  ...,  K (excluding  I)  are  used  to  represent  the  ten  k'  in- 
put variables  Xi,  Using  the  identifying  contrast  shown  In 

Table  11  yields  the  design  matrix  given  In  Table  12. 
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Table  10.  "Minimum  Number  of  Points"  Design  Matrix 
for  IS  Variables 


1 


1 

000000000000000 

96 

2 

loooooooobooooo 

97 

3 

OinOOOODODOOOOO 

98 

9 

0010000  0000  00  00 

99 

5 

000100000000000 

so 

6 

000010000000000 

51 

7 

000001000000000 

52 

8 

000000100000000 

53 

9 

QOOOOOQIOQQOOOO 

59 

10 

000000001000000 

55 

11 

a 00 0000 DO 10 00 00 

56 

12 

QOOOOnOOQOlOOOO 

57 

13 

000000000001000 

58 

19 

000000000000100 

59 

IS 

QOOODOOqOOOQOlO 

60 

16 

000000000000001 

61 

17 

200000000000000 

62 

18 

020000000000000 

63 

19 

002000000000000 

69 

20 

000200000000000 

65 

21 

OOQ02QOQOOOQOOO 

66 

22 

000002000000000 

67 

23 

000000200000000 

68 

29 

000000020000000 

69 

25 

000000002000000 

70 

26 

0 00  00  0 0 00  2 0 00  00 

71 

27 

000000000020000 

72 

28 

000000000002000 

73 

29 

000000000000200 

79 

30 

OOQOOOOOOQOQO 20 

75 

31 

000000000000002 

76 

32 

iinooQooooooooo 

77 

33 

lOlOOOOQQOOOOOO 

78 

39 

100100000000000 

79 

35 

100  01  00  00  00  00  00 

80 

36 

IQOOQIQOOOOOQOO 

81 

37 

lOOOOOlOQOOQOQO 

82 

38 

lonooooiooooooo 

83 

39 

100000001000000 

89 

90 

100000000100000 

85 

91 

100000000010000 

86 

92 

10000000000 1000 

87 

93 

100000000000100 

88 

94 

100000000000010 

89 

95 

looonoooooooooi 

90 

oii^oboooooonoo 
0 10 10  QD  00  00  00  00 
010010000000000 
010001000000000 
010000100000000 
0 10 00  00 10  00  00  00 
01000  00  01 00  GO  00 
0 10  QQ  00  00 10  00  00 
0 1000  00  0001  0000 
010000000001000 
010000000000100 
OlOOQQOOOOOOOlO 
010000000000001 
001100000000000 
0010100  0000  0000 
001001000000000 
0010001  0000  0000 
00100001000  0000 
00100  00  0100  0000 
0010000001000  00 
001000000010000 
0010000  00  001000 
001000000000100 
00100000000  0010 
001  00  00  00  00  00  01 
000110000000000 
0 0010100000  0000 
0001001 0000  00  DO 
000100010000000 
000 10  on  01 00  00  00 
0001000  00100000 
0 001000  0001  0000 
0 001000  00  001000 
000 10  00  00  00  01 00 
0 0010  00  00  00  0010 
000100000000001 
000  01 10  00  00  00  on 
0000101  00000000 
0 bo  01 00 10  00  00  00 
0 000100  01000000 
0000100  00100000 
0000100  0001  0000 
000010000001000 
000010000000100 
0000100  00  00  0010 


BOObfOBfUaHOfiBSi 

ooooouoooooooo 
000  0010100000  00 
000001001000000 
0 00001000100000 
0 00001000010000 
000001000  001000 
0 00001000  000100 
000001000000010 
0 00001000000001 
0 00000110000000 
0 00000101000000 
000  00  01 00 10  00  00 
000  00  01 00  01 00  DO 
000000100001000 
onoo  01 00  000100 
0 00000100000010 
0 00000100000001 
0 00000011000000 
0 00  00  00  10  ID  00  00 
0 00  00  00  1001  00  00 
0 00  00  00  10  00  10  00 
0 00  DO  00 10  00  01 00 
0 00000010000010 
000000010000001 
0 00000001100000 
0 00000001010000 
0 00  00  0001  001000 
0 00  00  0001000100 
0 no  00  DO  01 00  00 10 
0 00  00  0001000001 
0 00  00  00  DO  11 00  00 
0 00000000101000 
0 00000000100100 
0 00000000100010 
0 00000000100001 
000000000011000 
00000  0000010100 
0 00000000010010 
0 00000000010001 
00000  0000  001100 
0 0000  0000001010 
0 00000000001001 
0 0000  0000000110 
0 00000000000101 
0 0000  0000000011 
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93 
99 

95 

96 

97 

98 

99 
100 
101 
102 
103 
109 
108 
106 

107 

108 

109 

110 
111 
112 
113 
119 

115 

116 

117 

118 

119 

120 
121 
122 
123 
129 

125 

126 

127 

128 

129 

130 

131 

132 

133 
139 

135 

136 
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labia  12.  3^^"^  Fractional  Factorial  Design  Matrix 
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1 

3 

:« 

4 

n 

oooodboooo 

0003ri3030 

00013~31010 

0101010039 

OlOOlliOOlO 

1 

3130019030 

3133131010 

3131300000 

3331033013 

3330101009 

9 

1310031010 

1913100000 

1911313030 

1011001003 

1010110033 

|| 

0119110313 

0111333309 

0110001393 

0310130301 

0913303931 

3903139303 

9301301933 

3300010913 

3000103331 

3003311311 

'1 

1033101393 

1031310919 

1030033303 

1130111311 

1133830901 

'1 

0831330191 

0330008111 

0833111101 

0033900110 

0091013100 

■'u 

9011303111 

3010011101 

3013190131 

3113913100 

9111031130 

1101311101 

1100080181 

1103103111 

1309331190 

1901000110 

‘■il 

S 

7 

8 

3 

10 

0109801003 

0909080011 

0301109001 

0900811031 

0109130100 

,1 

3393310038 

3099003001 

9031111031. 

3090380011 

3333103130 

1013333013 

1119011081 

1111190011 

1110909001 

1013111110 

0911011311 

0011100990 

0010819910 

0019091900 

0311300013 

9001090301 

9101113910 

3100391900 

3109000380 

3001813009 

'■'1 

1181009381 

1881191300 

1380300390 

1339019310 

1131931083 

0090191190 

0190910103 

0133098139 

0191101119 

0090010931 

V- 

9110100110 

9810989189 

9919001119 

9811110103 

3110033311 

-f 

1900118100 

1000901118 

1009010103 

1001133198 

1900001901 

■'[j 

11 

19 

13 

14 

IS 

i 

0101308180 

0100011110 

0900100189 

0308918113 

0301031103 

■V 

3891811110 

8890090100 

3080113113 

3083331103 

3091000199 

1011990100 

1010009190 

1110181109 

1118900188 

1111019119 

! 

0810019009 

0918191099 

0019310001 

0011099031 

0010101011 

9000091099 

9009100019 

9109399091 

9101001011 

3100110001 

1190000019 

1198118009 

1998901011 

1831010001 

1830199091 

. 

0088199911 

0081901901 

0191090910 

0180108900 

0198911890 

1 

9119101901 

9111910991 

8911008900 

9910111930 

9919390910 

j 

1803110391 

1901999911 

1001011990 

1000190910 

1003308800 

!£ 

17 

18 

13 

30 

\ 

0001110111 

0000393101 

0003001131 

C801910300 

0300698330 

9181133101 

3130301131 

31330101U 

3031333380 

3090001910 

) 

rV 

1311101131 

1310310111 

1318033101 

1111901810 

1110010900 

0110330030 

0113003010 

OlllUlOOO 

0010030113 

0013103109 

1 

3300303010 

3303011000 

9301130030 

9100009103 

9108111193 

1080911000 

1033090030 

1031103010 

1330011138 

1933190119 

0933000303 

0331113333 

0330331313 

0139100031 

0131913011 

3013013333 

3011131313 

3010800903 

3919113011 

9911931001 

1103031313 

1101100303 

1100319333 

1003131001 

1001300031 

31 

U 

0303101310 

0003330333 

0001003913 

0000111303 

3033110300 

3133303313 

3131011303 

3130130333 

1113133330 

1313311303 

1311030333 

1910103913 

0011311133 

0111000101 

0110113131 

0113331111 

3101330113 

3301019131 

3300131111 

3309300101 

1331303103 

1031031111 

1030100101 

1093313131 

0130031001 

0330110010 

0333333000 

0331001030 

3910000031 

3010133000 

3019301030 

3011010010 

1000013011 

1100101030 

1103310010 

1101033000 

36 

37 

0100300311 

0103013301 

0101131331 

3330313301 

3333031331 

3331100311 

1010331331 

1013000311 

1011113301 

0313010130 

0311133110 

0310301100 

3003033110 

3001101100 

3000310130 

1133001100 

1131110130 

1130333110 

0031190003 

0030303033 

0033011013 

3111103033 

3110311013 

3113030003 

1301111013 

1300330003 

1309003033 
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The  design  permits  fitting  and  testing  20  single-variable  terms  and 
180  Interaction  terms.  The  20  single-variable  terms  are  the  10  Alnear 
terms  xi,  xo,  ....  Xin  and  the  10  quadratic  terms  xf,  xi,  ...,  xfn. 

Each  of^the^(lOx9)/2'‘*  45  Interaction  effects  have  four  terms, 

?1^1'  giving  a total  of  180  two-variable  terms.  This 

Uam  42''degretts  of  freedom  for  Lack  of  Fit  which  can  be  used  for  test- 
ing. The  fitted  function  of  each  measure  of  effectiveness  can  then  be 
studied  to  determine  the  optimum  systems  mix  In  the  same  manner  ex- 
plained In  the  previous  section. 


6.  SUMMARY.  Four  candidate  experimental  designs  have  been  pro- 
posed for  the  Target  Acquisition  Systems  Force  Mix  Evaluation  Analysis 
methodology  development.  The  designs,  two  two- level  designs  and  two 
three-level  designs,  are  recommended  for  sequential  application.  The 
resolution  III  and  resolution  V two-level  deslgos-contaln  16  and  256  de- 
sign points,  respectively.  The  MNOP  and  the  fractional  factorial 
three-level  designs  contain  136  and  243  design  points,  respectively. 
Therefore,  four  candidate  combinations  of  2’’’  and  sequential  designs 
are  available.  The  total  number  of  computer  model  simulation  runs  range 
from  16  + 136  ■ 152  to  256  + 243  ■ 499.  The  sequential  designs  and  the 
required  numbers  of  model  runs  are  shown  In  Table  13.  The  number  of 
model  runs  Is  shown  In  parentheses. 


Table  13.  Candidate  Sequential  E"’  and  3"  Designs 


Two  level  Three  level 
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Environmental  and  Water  Quality  Operational 
Studies:  Experimental  Design  Problems 
Associated  with  the  Fisheries  of  the 
Mississippi  River 


Michael  F.  Farrell, 

A.  Dale  Hagoun, 

Enviroxunental  Laboratory 
Waterways  Experiment  Station 
Vicksburg,  Mississippi 

Introduction 

The  Waterways  Experiment  Station  is  conducting  a six-year,  nationwide 
program  of  applied  research  to  investigate  sslectsd  high  priority  environ- 
mental quality  problems  associated  with  the  Civil  Works  sctlvltlsa  of  the 
Corps  of  Engineers  (CE).  The  study  is  being  conducted  for  the  Office, 

Chief  of  Engineers,  and  is  entitled  the  Environmental  and  Water  Quality 
Operational  Studies  (EWQOS). 

The  principle  goal  of  EWCOS  is  to  provide  new  or  Improved  methodologies 
and  technology  for  the  planning,  design,  construction,  and  operation  of  CE 
projects  to  meat  environmental  quality  objectives  in  a manner  compatible 
with  project  purposes.  A key  element  of  EWQOS  is  the  use  of  extensive 
field  studies  to  evaluate  and  document  the  utility  of  new  or  improved 
methodologies  and  technology  developed  within  the  program. 
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During  the  early  planning  phasa  of  EWQ0S»  via its  by  WES  parsonnal 
ware  nada  to  aach  CONUS  Corps  Division  office  to  identify  and  aasass  the 
magnitude  of  anvlronaantal  quality  problema.  The  nature  and  extant  of 
environmental  quality  problems  being  experienced  by  Corps  flsld  offices 
was  used  as  a basis  for  planning  research  to  address  these  problems. 

One  major  problem  area  Identified  ae  being  of  high  priority  was  the 
environmental  impacts  of  project  activities  on  waterways.  Speeiflcally» 

It  was  detacmlned  that  EWQOS  research  should  develop  field  office  guid- 
ance to  address  anvironmental  and  water  quality  aspects  of  dikes  and 
revetments.  Such  structures  are  common  in  waterways  in  many  parts  of 
the  United  States , but  most  commonly  occur  along  the  Mjssissippl  River 
and  Its  tributaries. 

Based  on  this  problem  identification  phase,  a project  was  established 
within  EWQOS  to  conduct  a comprehensive  field  study  of  dikes  and  revet- 
ments associated  with  CE  waterway  navigation  projects.  This  field  study 
Is  being  undertaken  within  a 50  mile  reech  of  the  Lower  Hlsslssippi  River 
between  Lake  Providence,  Louisiana,  and  Greenville,  Mississippi.  This 
site  was  selected  after  an  intensive  survey  of  CE  waterways  navigation 
projects  end  efter  discussions  with  knowledgeable  CE  field  personnel. 

Site  selection  was  based  on  the  existence  of  an  extensive  hydraulic  and 
hydrologic  data  base,  the  presence  of  a representetlve  variety  of  dike 
and  revetment  structure  design,  and  optimum  diversity  of  characteristic 
floodplain  and  riverine  aquatic  nacrohabltats , and  plans  by  the  Vicks- 
burg District  to  conduct  potsmology  studies  in  the  reach  during  thd  time 
frame  of  EWQOS,  and  to  synthesize  in  a report  the  existing  hydraulic  and 
sediment  data. 

Goals  and  Objectives 

The  goal  of  the  long-term  waterway  field  study  of  dikes  and  revet- 
ments is  to  assess  the  relative  ecological  importance  of  channel  align- 
ment and  bank  stabilization  structures  in  the  riverine  ecosystem,  and 
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to  provide  data  to  formulate  environmental  quality  guidelines  for  use  by 
CE  districts  In  deslgn.tng  and  planning  new  structures  and  modifying  exist- 
ing ones. 

The  specific  objectives  are  based  on  a macrohabltat  approach  in  which 
revetted  banks  and  dike  fields  are  considered  as  aquatic  habitats  within 
the  river  system. 

(1)  Quantitatively  define  riverine  macrohabitats  of  the  study  reach 
including  relative  site,  current  velocity,  sediment  type,  materials  compos- 
ing the  dikes  and  revetments  and  associated  riparian  vegetation  at  various 
rivet  stages  and  tines  of  year. 

(2)  Quantitatively  describe  the  physicochemical  characteristics  of 
the  water  and  sediments  in  riverine  nacrohabitats  at  various  river  stages 
and  times  of  year  and  how  these  variables  relate  to  the  distribution  and 
abundance  of  aquatic  organisms. 

(3)  Quantitatively  describe  the  composition  of  the  particulate 
organic  matter,  including  phytoplankton,  xoop‘’tnkton,  and  detritus.  In 
riverine  macrohabitats  at  various  river  stages  and  times  of  year. 

(d)  Quantitatively  describe  the  species  diversity,  abundance  distri- 
bution and  production  of  benthic  macroinvertebrates  in  riverine  macro- 
habitats,  Including  the  use  of  these  habitats  as  spawning,  nursery,  and 
feeding  areas. 

<5)  Quantitatively  describe  the  species  diversity,  abundance 
distribution  and  production  of  fishes  in  riverine  macrohabitats.  Includ- 
ing the  use  of  these  habitats  as  spawning,  nursery  and  feeding  areas. 

General  Background 

Dikes  have  been  placed  In  rivers  by  the  CE  for  many  years  for  the 
purpose  of  aligning  and  contracting  river  channels.  The  lower  Mississippi, 
middle  Mississippi,  Missouri,  and  Arkansas  are  examples  of  rivers  that 
have  extensive  amounts  of  dike  structures.  In  navigation  projects,  the 
principle  use  of  dikes  Is  In  adjusting  channel  width,  depth,  and  align- 
ment and  to  close  secondary  channele  end  chutes.  Dike  structures  are 
probably  the  most  effective  naans  of  channel  alignment  and  contraction 
in  use  today. 

Dikes  are  structures  constructed  of  permeable  wooden  piles  or,  more 
typically  In  present  times,  of  relatively  impemable  stone  riprap.  Dikes 
may  be  singular  or  placed  one  after  another  along  a bank  forming  c dike 
field.  Generally  dikes  are  of  the  transverse  type  which  extend  from  the 
bank  perpendicularly  into  the  river  channel  past  the  point  of  highest  cur- 
rent velocities.  An  extension  or  L-head  may  be  placed  at  the  off-shore 
end  of  a dike  parallel  to  the  main  axis ’of  the  dike  to  retard  scouring 
and  turbulence.  Vane  dikes  which  are  placed  in  the  channel  parallel  to 
the  bank  line  are  also  used. 
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Water  Is  shunted  by  a dike  toward  the  opposite  river  bank  and,  If 
this  bank  Is  stable,  the  resulting  narrower  channel  Is  deepened  by  scour- 
ing In  order  that  the  river's  discharge  may  be  carried.  Dikes  are 
typically  placed  on  the  convex  aide  or  point  bar  in  a bendway  or  in 
straight  reaches  to  achieve  some  degree  of  channel  sinuosity  and  result- 
ing contraction.  Concurrent  with  channel  contraction,  suspended  sediments 
are  deposited  downstresn  of  Individual  dikes  due  to  the  reduction  of  cur- 
rent velocities  caused  by  the  structure.  In  dike  fields  sediment  accretion 
may  be  appreciable,  and  these  accumulated  sediments,  which  may  In  some 
Instances  form  fast  land  or  a bar,  further  serve  to  confine  the  flow  of 
water.  Slackwater  pools  may  be  found  downstream  of  transverse  dikes  at  low 
river  stages  In  cases  where  sediment  accretion  has  not  completely  filled  In 
these  areas.  Since,  ecologically,  slackwater  areas  are  thought  to  be  im- 
portant In  a river  system,  knotches  have  been  put  In  transverse  dikes  to 
prevent  sediment  buildup  on  the  downstream  side  of  dikes. 

In  the  lower  Mississippi  River  there  are  presently  393  dikes  totaling 
approximately  875,000  linear  feet  (Table  1).  The  number  of  dikes  dimin- 
ishes downstream  In  lower  Mississippi  Fiver,  with  no  dikes  being  present  in 
the  river  within  the  confines  of  the  Mew  Orleans  District.  However,  many 
new  structures  are  planned  within  the  next  two  decades  in  the  lower  river. 

Despite  the  large  number  of  dikes  present  In  many  of  the  major  river 
systems  of  the  United  States,  the  ecological  effects  of  these  structures 
are  poorly  known.  Dike  fields  and  individual  dikes  are  distinct  habitats 
within  river  systems  where  these  structures  are  numerous.  Date  on  this 
environmental  quality  characteristics  of  these  structures  and  methods  for 
designing  and  modifying  dikes  to  enhance  their  value  as  aquatic  habitat  is 
needed  by  CE  districts  and  divisions  In  designing  and  operating  waterway 
navigation  projects. 

Revetments  are  Installed  along  river  banks  to  prevent  bank  caving  and 
erosion.  These  structures  are  of  many  types,  but  generally  consist  of 
erosion-resistance  materials  which  are  placed  upon  a pregraded  bank  from 
the  top  of  the  bank  line  to  the  toe  of  the  channel.  In  navigation  and 
flood  control  projects  revetments  are  often  located  on  the  concave  bank  In 
bendways  and  crossings  In  association  with  dike  fields  on  the  opposite  bank 
to  contract  the  river  channel  and  to  retard  meandering.  They  are  also 
placed  In  areas  where  erosion  threatens  levees. 

Bank  revetments  are  of  many  different  types  including  stone  riprap 
and  articulated  concrete  mattresses  (ACM),  ACM  with  stone  riprap  placed 
on  the  upper  portions  of  the  bank  as  paving  Is  the  moat  common  type  of 
revetment  presently  being  Installed  in  the  lower  Mississippi  River.  In 
the  past,  asphalt  and  willow  mats  were  used  Instead  of  riprap  for  bank 
paving.  Approximately  3.9  million  linear  feet  of  bank  revetment  works 
have  been  constructed  In  the  lower  Mississippi  River.  In  the  Missouri 
River,  revetments  constructed  of  stone  riprap  are  used  extensively. 

When  river  banks  are  revetted,  much  of  their  natural  character  Is 
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altered.  However » the  environmental  quality  of  the  habitat  formed  by  the 
revetment  atructurea  la  unknown.  Older  revetment  structures  which  have 
become  extensively  vegetated  with  willow  and  cottonwood  trees  and  a variety 
of  sedges,  grasses,  and  shrub a may  have  different  value  as  aquatic  habitat 
than  new  revetments.  Thus  age  is  a variable  of  importance  In  evaluating 
environmental  effects  of  revetments.  The  type  of  material  composing  the 
revetment  may  also  be  an  important  variable  In  determining  the  biological 
productivity  and  habitat  value  of  revetment  structures.  For  exsmpla, 
stone  riprap  la  probably  a more  productive  substrate  for  fish  food  organisms 
than  asphalt.  Another  factor  Is  tha  sinuosity  of  the  bank  line  upon  which 
revetment  Is  placed.  A sinuous  bank  would  tend  to  have  relatively  lower 
velocities  caused  by  addles  and  upstream  flow  than  would  a straight  bank 
line,  and  would,  therefore,  be  expected  to  constitute  more  productive  fish 
habitat. 


The  large  magnitude  of  revetted  banks  in  many  river  systems  make  Inves- 
tigation of  the  environmental  effects  of  these  structures  of  value  to  CE 
districts  and  divisions  for  use  In  Impact  assessments,  and  the  planning, 
design,  and  construction  of  new  structures.  At  present  there  exists  vary 
little  environmental  data  on  revetment  structures  regarding  their  produc- 
tivity as  compared  to  ’'natural"  banks. 

The  distribution  and  abundance  of  organisms  In  a large  river  system 
are  fundamentally  datarmlned  by  river  geomorphology,  flow,  and  sediment 
load  for  It  Is  these  factors  which  Interact  to  produce  distinct  environ- 
ments or  macrohabitats  for  riverine  biota.  Undoubtedly,  water  quality, 
temperature  sediment  type  and  other  factors  are  important  determinants  of 
the  distribution  and  abundance  of  aquatic  species  In  a given  river  but 
these  are  secondary  to  the  more  basic  geomorphology  and  hydraulic  features 
of  the  stream.  Land  use  practices,  terrestrial  vegetation  patterns,  and 
adaphlc  characteristics  In  the  drainage  basin,  also  contribute  importantly 
to  the  environmental  conditions  of  the  river,  but  these  factors  mainly 
impose  conditions  upon  the  macrohabitats  formed  by  the  rivers'  shape, 
flow,  and  sediment  load.  For  example,  a broad,  shallow  braided  river  has 
different  proportions  and  types  of  macrohabltats  than  a stream  with  a deep, 
narrow  well-defined  channel,  Irrespective  of  water  quality. 

The  channel  alignment  and  bank  stabilisation  structures  placed  In 
rivers  by  the  Corps  for  achieving  flood  control  and  navigation  objectives 
can  modify  to  different  degrees  geomorphology,  stage  and  discharge  rela- 
tionships, and  aedlmant  movements  within  tha  stream.  These  changes  in 
the  river's  characteristics,  plus  the  presence  of  the  structures  them- 
selves as  a substrata  for  organisms,  result  In  shifts  In  the  types,  slsas 
and  variety  of  aquatic  macrohabltats.  Such  alterations  In  riverine  habitats 
effected  by  the  Installation  of  structures  may  or  may  not  produce  positive 
Impacts  on  the  ecology  of  the  system.  For  example,  certain  "natural"  macro- 
habltats may  be  reduced  In  sise  and  quality  by  training  and  stabilization 
structures,  while  habitats  created  by  the  Installed  structufea,  such  as 
dike  fields  or  revetted  banks,  may  become  commonplace.  The  primary  goal  of 
this  research  la  to  determine  the  ecological  Importance  or  value  of  the 
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nacrohabltacs  formed  in  the  river  by  dike  fields  and  revetted  banks  as 
compared  to  "natural"  habitats,  and  to  formulate  environmental  quality 
guidelines  for  use  in  designing  and  planning  new  structures  and  modify- 
ing existing  onas. 

The  gsnaral  approach  to  the  dike  and  revetment  study  In  the  lover 
Mlseiselppl  River  will  be  from  a macrohabitat  point  of  view,  wherein  the 
length  of  river  to  be  inveetlgated  will  be  eubdivlded  into  ite  macro- 
habitats  which  will  be  characterised  as  to  both  biotic  and  abiotic  vari- 
ables. Dike  fields  and  revetted  banks  will  be  treated  as  macrohabitats, 
although  man-made,  together  with  other  distinct  biological  habitats  such 
as  the  main  channel,  natural  steep  banks,  towhaads,  chutes,  river  borders, 
point  bar  cutoffs,  and  old  river  channels.  The  ecologlcel  Importance  of 
each  maerohabltat  will  be  evaluated  according  to  its  basic  water  quality, 
production  of  benthic  organisms  used  as  fish  food,  abundance  of  fishes, 
use  as  spawning,  nursery  and  feeding  areas  for  fishes,  and  production  of 
suspended  particulate  organic  matter.  The  function  and  relative  value  as 
aquatic  habitats  of  dike  fields  and  revetted  banks  within  the  river  eco- 
system will  be  defined  based  on  this  information.  The  habitat  evaluation 
will  ba  related  to  various  key  river  stages  and  times  of  year. 

The  dike  and  revetment  study  will  ba  initiated  by  preparing  a quanti- 
tative nap  of  aquatic  macrohabltata  in  the  50  mile  reach  of  lower  Miss- 
issippi River  selected  for  study.  These  data  will  form  the  basis  of  the 
field  investigations.  The  map  will  be  prepared  using  existing  hydraulic 
and  topographic  survey  data  and  aerial  photographs  belonging  to  the 
Vicksburg  District  as  a basis.  Habitats  will  be  initially  delineated  at 
a low  water  stage,  defined  as  0 ft  Low  Water  Reference  Plane,  using  as 
criteria  primarily  depth  and  information  from  aerial  photographs  as  to 
the  location  of  sandbars  and  other  features.  This  map  will  be  refined  as 
to  the  definition  of  habitats  with  data  on  currents,  sediments,  end  biota 
collected  during  the  pilot  survey.  The  map  will  be  redefined  for  bank 
full  and  over-bank  river  stages. 

The  first  efforts  in  the  field  will  be  a series  of  pilot  surveys 
designed  to  provide  additional  data  for  the  habitat  map,  for  developing 
and  tasting  sampling  equipmant  and  techniques,  selecting  representative 
habitats  for  intensive  study,  and  developing  an  experimental  design  for 
the  habitat  surveys  and  dstalled  ecological  studies.  The  pilot  survey 
will  be  conducted  from  April  through  September,  1978. 

Following  completion  of  the  pilot  survey  and  formulation  of  experi- 
mental designs,  at  least  two  representatives  of  each  habitat  type  will 
be  selected  for  comparative  study.  Dike  fields  and  revatmants  will  be  in- 
cluded in  the  category  of  habitate.  Selected  habitats  will  be  surveyed 
intensively  four  times  each  year,  beginning  in  the  fall  of  1978.  Data  on 
biota,  water  and  sediment  chemistry  and  physical  variables  will  be  collected 
in  a manner  amenable  to  detailed  statistical  comparisons.  The  four  yearly 
samples  will  be  related  to  distinctive  river  stages  and  seasons. 
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RED  NOISE  IN  THE  POWER  SPECTRUM  OF  ATMOSPHERIC  TEMPERATURE  DATA 


Oakar  M.  Baaamrangar 
Raaaarch  Dlractorata 
Vaehnologjr  Laboratory 
US  Army  Mlaalla  R&D  Comnand 
Radatona  Araanal.,  AL  35809 


ABSTRACT.  It  la  vail  known  that  paralataneo  ganarataa  rad  nolaa  In 
powar  apaetra  of  mataorologleal  timo  aariaa.  In  fact,  algnlflcanca  of 
apaetral  paaka  la  ohackad  agalnat  tha  background  of  white  or  rad  nolaa, 
although  othat  criteria  hava  baan  auggaatad  In  tha  lltaratura. 

Savaral  typaa  of  rad  nolaa  axlat.  In  atmoapharlc  aclanca  the  moat 
cotmon  typa  la  tha  axponantlal  modal  which  la  Identical  with  tha  flrat 
order  Markov  chain.  Thla  typa  la  aaay  to  raeognlaa  and  can  be  readily 
calculated  from  tha  corrclogram. 

Today  power  apectra  are  moatly  produced  with  the  algorithm  of  the 
Feat  Fourier  Tranaform  (FFT)  rather  than  baaed  on  the  autocorrelogram. 
Conaaquently,  calculation  of  tha  exponential  rad  nolaa  la  difficult  unleaa 
the  flrat  lag  correlation  la  Included  In  tha  computer  procaaalng. 

Two  queatlona  have  baan  puraued  In  thla  paper.  What  la  the  effect 
on  tha  escponentlal  rad  nolaa  pattern  If  any  periodicity,  algnlf leant  or 
not,  la  removed  from  the  power  apectrum?  Can  true  parlodlcltlea  with 
low  wave  numbera  be  dlatlngulahed  from  rod  nolaa? 

The  Influence  of  "quaalperiodiclty"  on  red  nolaa,  the  power  apectrum 
and  Fourier  componenta  la  analyxod  for  atmoapharlc  temperature  date. 
Finally,  the  utlllaatlon  of  dlacrete  apectral  flltara  will  be  diaeuaeed 
and  a aopat'atlon  of  the  time  aerlea  Into  cyclea,  quaalcyclea  and  red  nolaa 
la  delineated. 

I.  INTRODUCTION.  The  queatlon  of  the  reality  of  meteorological 
cyclea  haa  bean  ralaod  at  varloua  tlmea  In  the  peat  (e.g.,  Bartela,  1943; 
Brier  at  al.,  1964;  Craddock,  1965}  Shapiro,  1975}  etc.)  and  haa  not  bean 
completely  aettled.  The  phyalcal  mechaniam  behind  the  annual  and  dally 
cycle  In  meteorological  time  aerlea  leavea  no  doubt  about  their  reality, 
but  moat  othar  parlodlcltlea  are  accepted  by  aoma  authora  and  rejected  by 
othera.  Although  algnlflcanca  criteria  have  bean  developed  In  vtatlatlcal 
analyala,  the  aubjactlvlty  In  the  adoption  uf  a algnlflcanca  threahold 
leavea  aome  ambiguity. 
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la  cddltloa,  many  criteria  are  baaed  on  the  poatulatlon  that  the 
data  are  Independent  and/or  Oauaalan  diatrlbuted  vhlch  la  not  applicable 
to  toany  atmoepherlc  elemente.  It  la  veil  known  that  moat  atnoepherlc 
time  aerlea  ahov  varying  degreea  of  peralatancei  Thla  peralatence  la 
uaually  taken  Into  account  by  teating  time  aerlea  datai  a>g.|  power  apectra, 
againat  "red  nolae"  which  la  generated  by  peralatence • An  added  difficulty 
la  the  appearance  of  the  "quaalparlodlclty"  where  epparent  cyclee  prevail 
during  a limited  time  only. 

It  la  well  known  that  red  nolae  la  related  to  the  flrat  lag  cor- 
relation coefficient  but  the  modlflcetlon  of  thla  coefficient  by  the 
preaenee  of  a cycle  la  neglected  In  moat  referencea.  It  will  be  ahown 
that  the  conatructlon  of  red  nolae  ie  not  Independent  of  the  preaenee  of 
cyclee,  and  It  la  not  trivial  to  aalect  a red  nolae  teat  backgrotmd. 

Time  aerlea  data  of  atmoapharic  temperature  aerve  aa  an  example  for 
a mixture  of  cyclee  and  red  nolae  pattern.  An  attempt  la  made  to  eeparata 
theae  two  eomponenta.  It  proved  that  the  power  apectrum  haa  the  advantagea 
of  dlacloalng  algnlflcant  cyclea  In  concentrated  form  and  of  reapondlng 
better  to  quialperlodlcity  than  the  Fourier  analyala.  The  Fourier  analyala 
la  the  moat  appropriate  tool  to  provide  quickly  amplitude  and  phaee  angle 
for  apectral  filters.  The  periodogram  analyala  ravaala  quaalparlodlclty 
beat  and  permlta  ua  to  pinpoint  the  exact  cycle  length. 

II.  THE  EXPOOTMTIAL  MD  NOISE  MODBL.  Different  typea  of  red  nolae 
pattema  iiave  i>een  dlacueaad  by  the  author  (1977),  and  only  the  equationa 
for  the  exponential  model  will  be  preaented  here. 

The  moat  common  red  nolae  In  meteorology  follova  a plain  exponential 
aequancet 

Pj,  - exp(-bt)  (1) 

with 

t ^ 0,  b > 0, 

This  exponential  aerlea  la  alao  obtained  from  e flrat  order  Markov 

chain: 

P|j  - P*^.  (2) 

It  la  trivial  that  t ■ k,  b ■-An  p,  and  0 < p ^ 1.0.  In  the  flrat  order 
Markov  chain  p p. . The  conatant  b In  the  exlponentlal  nolae  can  be  cal- 
culated from: 

b ■ -Inpj^  (3a) 


W«  Duat  also  k««p  in  mind  that  by  aqn.  (1)  tha  p^followa  tha  axponantial 
dlatribution  (aaa  Baoanvangar,  1976,  p.  113) « Conaaquantly: 

|p^  - l/bj^  (3b) 

Bacauaa  in  practical  applicatlona  tha  suonation  la  truncatad  aftar  m tarma, 
b bj^,  and  aoma  dlaorapanclaa  batmaan  aqn.  (3a)  and  (3b)  may  arlaa. 

A.  aultabla  aquation  for  tha  povar  apeetrum  haa  baan  daducad  by 
Oilman  at  al.  (1963).  Thay  oaati 


L,  - Cl  - p^)/<l  - - 2p  eoa  kir/mQ/m  (4) 

In  uur  caaa  p s p^. 

Othar  modala  and  thalr  dlffarancaa  hava  baan  analyaad  by  tha  author 
(1977).  It  baoomaa  avldant  that  p.  la  a crucial  paramatar  in  rad  nolaa 
analyala.  Eqna.  (1)  and  (4)  ara  rflatad  byi 

m 

p,.  ■ EL.  eoa(tk7r/m)  (5) 

* k-l* 


III.  RED  NOISE  HIXTURB  WITH  CYCLES . Tha  ganaral  mataorologleal  tima 
aarlaa  may  eomprlaa  a mlxtura  o^  rad  nolaa  and  pariodicltiaa.  Thia  fact 
waa  pravioualy  axampllflad  with  tha  aid  of  thraa-hourly  tamparatura  data 
at  Huntavllla,  Alabama  for  a tima  parlod  from  6 Novambar  195B  through 
14  March  1939  with  N ■ 1024  (aaa  Saaanvangar,  1977).  In  thia  artlcla 
tha  author  haa  llluatratad  that  an  appropriata  combination  for  tha  atructura 
of  tha  autocorralatlon  function  iat 


“iPl 


n 

E 

j-2 


(6) 


vhara : 


n 

E 

1 


(6a) 


and  pj^  donotaa  tha  rad  nolaa,  p^^^  tha  corralatlon  of  tha  raapactlva  cycla 

Lj.  (T&a  conaldaratlon  la  for  data  without  a trand).  Thia  almpla  llnaar 
(walghtad) combination  of  corralatlon  coafflclanta  vorka  battar  than  Flahar'a  a 
function  tdilch  la  auggaatad  for  tha  combination  of  corralatlona  by  varloua 
authora.  A numarlcal  axampla  which  conflrmad  aqn.  (6)  waa  glvan  by  tha 
author  (1977). 
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L«t  us  sssuins  that  u.  " 0.5  and  0^2  * 0.5.  The  first  lag  cor- 
relation coefficient  for  rid  noise  coula  be  ■•0.6  and  P2  ^ * 0.4 

from  a cycle.  Then  the  first  lag  correlation  of  the  data  aeries  is 
p|  ■ 0.5  (0.6  + 0.4)  * 0.5  which  is  less  than  p^  . In  this  csss  the 

correct  else  of  the  first  lag  corrslatlon  for  rei  noise  la  not  identical 
with  the  first  lag  correlation  of  the  data  series. 

IV.  POWER  SPECTRUM.  OOASIPBRIODICITY  AND  FOURIER  AMALTSIB.  Three 
different  statistical  tools  are  available  for  t^s  analysis  of  mstsoro- 
loglcal  time  aarlas  data:  power  spactrumt  Fourier  sarles  and  parlodo- 
gran.  These  three  Individual  techniques  serve  different  purposes. 
Consequently,  we  must  expect  that  different  characturlstlcs  are  enhanced 
by  the  analysis  of  the  same  data  sample  with  different  methods  although 
the  meteorological  characteristics  do  not  vary. 

A set  of  8-hourly  data  (one  year  from  1 September  1959)  was  sub- 
jected to  a Fourier  analysis.  Today  this  task  Is  rapidly  performed  by 
application  of  the  FFT,  the  fast  Fourier  Transform  (see  Cooley  and  Tuksy, 
1965).  Tha  FFT,  conveniently  performed  for  a basic  period  of  1024  ••2^^, 
shows  a first  significant  cycle  of  341.3  days,  the  annual  period. 

The  24^  cycle  la  not  a period  of  an  Integer  divisor,  and  is  spread 
over  the  wave  numbers  339-345.  A more  sophisticated  computer  program 
was  utilised  for  the  FFT,  which  does  not  raquire  that  tha  basic  period 
can  ba  expressed  as  ' power  of  2.  This  Fourier  analysis  was  established 
with  a basic  period  of  365  days.  In  this  analysis  the  day  is  an  integer 
wave  number,  and  a sharp  peak  appears  alone  at  24",  l.s,,  wave  number  365, 
without  spilling  over  to  the  adjacent  classes. 

This  fact  confirms  results  by  Rikllshl  (1976),  and  Illustrates 
some  fundamental  characteristics  of  power  spectrum,  FFT  and  Fourier 
analysis.  If  an  existing  cycle  Is  not  precisely  the  length  governed 
by  an  integsj:  wave  number  the  cycle  Is  "smeared"  over  the  neighboring 
frequencies.  One  solution  Is  the  use  of  a "filter  band"  (or  band  filter) 
whose  width  Is  determined  by  the  spreading.  If  one  particular  cycle 
exlste,  the  precise  length  can  ba  found  by  a perlodogram  analysis  (sea 
Essenwangar,  1976,  p.  234). 

The  utilisation  of  a filter  band  for  the  representation  of  a cycle 
has  some  added  benefits.  A peak  In  the  power  spectrum  may  indicate  quasl- 
perlodlclty.  This  sxpresaion  may  comprise  two  phsnomana.  First,  the 
cycle  length  may  fluctuate,  e.g..  In  our  case  between  5 to  6 days  (or  even 
4 to  7 days).  Second,  the  cycle  may  ba  repetitive  over  a few  periods, 
and  than  suddenly  either  disappear  or  become  longer  or  shorter.  The  power 
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ipactrum  or  Fourlor  analysla  raflacta  tha  cloaaat  mathanatlcal  fitting, 
raaambllng  an  avaraga  cyela  (aaa  Eitaanwangar,  1951).  Quaalparlodlclty 
la  vaakanlug  tha  amplltuda  ovar  tha  total  data  langth,  and  tha  algnlfl- 
canca  of  quaalparlodlclty  la  difficult  to  prova  by  atatlatlcal  taata 
hacauaa  of  tha  raaamblanea  to  random  eyclaa.  Indaad,  aoma  authora  con- 
aldar  quaalparlodlclty  by  and  largo  aa  a totally  random  product.  It 
ouat  bo  Intarjactad,  hovavar,  that  In  atmoapharlc  iclanea  a phyaleal 
background  for  quaalparlodlclty  can  aonatlmaa  ba  found.  B.g.,  a cyela 
of  5-6  daya  can  ba  ganaratad  by  tha  davalopnant  of  aata  of  eyclonaa, 
ao-callod  faalllaa  of  oyclonaa.  Tha  langth  of  thla  davalopnant  flue- 
tuatoa  but  tha  fact  of  Ita  axlatanca  cannot  ba  danlad. 

Tha  explanation  of  a 5-6  day  cycle  aa  related  with  tha  davalopnant 
of  cyclone  faalllaa  la  alao  aupportad  by  aa  oxanlnatlon  of  the  FFT  for 
tha  winter  1976/77  with  pradomlnantoly  aarldlonal  circulation.  In  thla 
"cold  winter"  at  Huntavllla,  Alabaaa.  the  5-6  day  "quaalparlodlclty"  waa 
abaanti  and  waa  replaced  by  an  8-9  day  quaaleycla. 

V.  ByMPIiE  OF  SEPARATING  RED  NOISE  FRCM  CYCl^S.  After  tha  utlllaatlon 
of  ilfiltar  ban^a  haa  iaan  explained  wa  return  to  tha  analyala  of  two  data 
aata  I tha  time  aarlaa  of  6-hourly  tanparatura  obaarvatlona  from  15  July 
1959  to  10  July  1960  and  15  July  1961  to  10  July  1962.  Tha  truncation 
of  5 daya  from  tha  yaar  almpllflad  tha  computational  afforta  for  tha 
application  of  tha  FFT  and  Ita  ralatlonahlp  with  tha  power  apactrum.  Later 
one  full  yaar  of  data  waa  utlllaad  but  tha  raaulta  from  tha  full  yaar  dif- 
fered only  by  O.IX  from  tha  truncated  aarlaa. 

Tha  elimination  of  cyclai  (quaiilcyeloa)  was  parfonaad  In  thrae  atapa 
(aaa  Flguraa  la  and  b).  Flrat,  tha  npaetrum  of  tha  original  data  aarlaa 
waa  obtained.  Then  the  annual  and  dally  cyclea  ware  removed.  Tha  remain- 
ing data  wara  aubjactad  to  tha  FFT  again.  A aarlaa  of  "quaalcyclea"  or 
"quaalparlodlcltlea"  vara  Identified  and  aubtractad.  Afterwardi,  tha 
apactrum  of  the  remaining  data  aarlaa  dlaplayad  only  Inalgnlfleant  de- 
viation from  the  racalculated  rad  nolaa  aarlaa  thua  leaving  tha  "nolaa" 
or  random  fluctuatlona  with  paralatanca.  Tha  cyclaa  and  quaalcyclea  with 
their  percentage  ahara  are  exhibited  In  Table  1. 

At  flrat  glance  wa  may  find  It  peculiar  that  the  annual  cycle  la 
Hated  aa  a flltar  band  from  tha  wave  numbara  1-7.  One  would  think  that 
tha  truncation  of  5 daya  from  tha  year  ahould  not  make  a algnlfcant 
dlffaranca.  In  reality  a peak  appeared  at  360  daya  with  a aide  lobe  at 
90  daya  langth.  It  waa  convenient  to  eliminate  tha  total  aarlaa  of  wavaa 
from  1-6  or  1-7.  Tha  appearance  of  thla  flltar  band  may  ba  cauaad  by  the 
aaynmatric  behavior  of  tha  annual  cycle. 
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Table  1. 


Separation  of  Temperature  Time  Series  Data 
into  Cycles  (Quasicycles)  and  Red  Noise 


1 

1 

1961 

1959 

1 

' Wave  Number 

X 

Wave  Number 

% 

I 

1 - 

7 

Annual 

67.0 

1 - 

6 

Annual 

71.3 

1 

12  - 

15 

24  - 30  days 

1.5 

13  - 

23 

16  - 28  days 

4.9 

1 . 
i* 

20  - 

28 

13  - 18  days 

3.9 

28  - 

33 

11  - 13  days 

1.7 

K 

1 

1 

33  - 

41 

9-11  days 

3.3 

43  - 

52 

7-8  days 

1.5 

51  - 

53 

7 days 

0.8 

61  - 

70 

5-6  days 

0.8 

1' 

73  - 

86 

4-5  days 

2.1 

- 

- 

- 

1' 

359  - 

361 

dally 

10.2 

359  - 

361 

dally 

8.6 

f' 

720 

semi-daily 

0.5 

710 

semi-daily 

0.4 

Red  Noise 

10.7 

Red  Noise 

10.8 

A distinct  peak  at  wave  number  360  appeared  which  signifies  the  dally 
cycle.  The  adjacent  classes  359  and  361  displayed  an  ampllnuJe  more  than 
10  times  as  large  as  the  adjacent  waves  (350  to  358  or  362  ';69).  In 

order  to  eliminate  a remaining  peek  at  the  daily  cycle  In  the  power  spectrum 
of  the  red  noise  series  the  band  from  359  to  361  was  removed. 


It  Is  evident  from  Table  1 that  the  "quaslcycles"  fluctuate  from  year 
to  year  which  should  be  expected.  These  quasiperlodlclties  vary  in  ampli- 
tude, phase  angle  and  duration.  As  pointed  out  some  authors  consider  them 
equivalent  to  random  fluctuatlone.  The  red  noise  series  and  displayed 
power  spectrum  after  removal  of  cycles  (middle  of  Figures  la  and  b) 
Illustrate  that  additional  cycles  (i.e.,  quaslcycles)  should  be  subtracted 
to  achieve  a better  agreement  (see  right  hand  side  of  Figures  la  and  b) . 


Thus,  the  time  series  of  the  two  sets  of  '‘.ata  samples  can  be  expressed 
by  3 cycles  (annual,  dally  and  semi-daily)  with  about  78  to  80!C  of  the  vari- 
ance, 4 or  5 (for  1959  and  1961,  respectively)  quaslcycles  with  an  added  9 
to  IIZ,  and  a remaining  red  noise  component  of  about  IIX.  The  proper  rad 
noise  Influence  is  then: 


% 


I ■.jPji 


/u. 


(7) 


where  ■ 0.86  for  the  1959/60  data  set,  n ■ 7,  and  0.59.  Tho  respec- 

tive numbers  for  the  data  set  1961/62  can  be  found  from^Flgure  lb  and  Table  1. 


It  Is  evident  that  the  true  red  noise  first  lag  correlation  Is  different  from 

Pi* 
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FIGTIBE  1.  SIX-HOURLY  TEMPERATURE  DATA 

HUNTSVILLE,  ALABAMA, 

(«)  15  JULY  1969  - 10  JULY  1990 
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We  can  safely  deduce  that  the  red  noise  pattern  cannot  be  obtained 
In  a trivial  way  In  the  presence  of  cycles.  Furthermore,  the  first  lag 
coefficient  p.  in  the  autocorrelation  la  not  automatically  the  one  which 
provides  the  proper  rad  nolae  entry.  How  does  this  result  affect  signi- 
ficance testing  of  the  power  apectrum  agslnat  the  statistical  background 
of  red  nolee?  The  answer  Is  not  simple  end  depends  on  the  goal. 

If  we  only  Intend  to  find  whether  the  power  apectrum  la  produced  by 
rad  noise,  the  pattern  based  on  the  first  lag  coefflent  may  suffice . If 
our  goal  Is  the  separation  of  the  data  eerles  Into  cycles  and  red  noise, 
a formalistic  application  of  the  first  lag  correlation  la  not  realistic 
unless  we  find  no  cycles. 

A similar  composite  pattern  to  the  autocorrelation  can  be  developed 
for  the  power  apectrum.  We  deduce: 


(8) 


where  stands  for  the  spectrum  of  the  filters:. 

The  reconstruction  of  the  red  noise  component  In  the  data  series 
la  not  trivial  because  the  phase  angles  for  the  Fourlor  terms  must  be 
known.  One  way  to  obtain  these  la  by  eliminating  the  filter  bands  from 
the  data  aeries,  and  subjecting  the  remainder  to  a FFT.  This  method  Is 
not  difficult  to  develop  once  It  is  known  which  cycles  (quaslcycles) 
must  be  removed.  Furthermore,  random  fluctuations  will  produce  candomnasa 
of  the  phase  angles  for  the  red  nolee  waves.  We  can  check  whether  the  re- 
maining phase  angles  display  randomness  because  It  requires  e rectangular 
distribution  of  the  angles.  The  result  Is  disclosed  In  Figures  2a  and  b. 
The  deviation  from  the  average  number  of  occurrence,  28.1  or  28.2,  was 
tested  for  statistical  significance  by  applying  the  Kolmogorov-Smlmov 
test  for  the  cumulative  distribution.  Nona  of  the  deviations  proved  to  be 
high  enough  to  reject  the  hypothesis  that  the  displayed  histogram  has  a 
rectangular  distribution  as  statistical  background.  We  find  confirmation 
that  the  remaining  data  series  behaves  like  red  noise  in  amplitude,  and 
now  In  phase  angles. 


yi.  CONCLUSICT.  The  background  of  red  noise  In  meteorological  time  series 
has  been  examined  for  temperature  data  at  Huntsville,  Alabama.  Of  special 
Interest  was  the  question  whether  significant  cycles  influence  the  determin- 
ation of  the  red  noise  pattern  from  autocorrelogram  or  spectral  analysis. 

As  illustrated,  the  presence  of  long  time  periodicities  tends  to  Increase 
the  first  lag  correlation.  In  fact,  any  existing  cycle  may  modify  It. 
Therefore,  the  judgment  of  red  noise  from  the  first  lag  correlation  alone 
may  be  Insufficient  in  many  cases. 
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This  result  is  important  In  the  evaluation  of  an  existing  time 
series,  but  it  may  be  of  little  consequence  for  the  testing  of  cycles 
against  red  noise  background.  However,  the  drawing  of  conclusions  from 
one  test  curve  alone  may  not  always  be  advisable.  A sequence  of  testing 
such  as  the  3 steps  Illustrated  In  Figures  la  and  b may  be  more  appro- 
priate. 

A separation  of  the  data  series  Into  rod  noise  and  cycles  wee 
delineated  (Figures  la  and  b).  This  method  resembles  Craddock's  (1965) 
suggestion  of  filtering  insofar  as  certain  algnlfleant  cycles  are 
filtered  out.  Two  significant  deviations  from  Craddock's  scheme  must 
be  emphasised.  First,  the  filtered  cycles  are  subtracted  from  the  total 
data  series  after  the  cyclas  have  been  determined  from  the  power  spectrum. 
Second,  the  remaining  data  series  Is  a red  noise  spectrum.  These  dif- 
ferences are  produced  by  the  dissimilarities  In  the  analysis  goals  be- 
tween Craddock's  method  and  the  technique  of  this  study. 

The  red  noise  contribution  to  the  variance  of  the  two  sets  of  data 
of  3 hourly  observation  for  about  one  year  Is  approximately  11%,  l.e., 
this  percentage  can  be  associated  with  random  noise  with  perslstencs 
while  the  remaining  flucuatlonc.  are  produced  by  cycles  or  quasicycles. 

About  80%  represent  the  diurnal,  semi-diurnal  and  annual  cycle  which  can 
be  considered  as  a simple  and  predictive  part.  Quasicycles  Impose  some 
restrictions  to  the  predictability  because  they  vary  from  year  to  year, 
and  the  red  noise  fluctuations  are  predictive  only  with  respect  to  their 
ststlstlcal  properties.  The  determination  of  the  contribution  of  red 
noise  in  meteorological  time  series  may  thus  be  Important  information. 
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SMALL  SAMPLE  BEHAVIOR  OF  SOME  PROCEDURES  USED 
IN  TIME  SERIES  MODEL  BUILDING  AND  FORECASTING 
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ABSTRACT.  This  paper  summariEea  the  results  of  a very  large  simulation 
study  of  some  procedures  commonly  used  In  time  series  model  building  and  fore- 
casting. Theoretical  results  available  in  this  area  are  generally  asymptotic 
and  exact  finite  sample  results  are  readily  obtainable  only  for  a few  over- 
simplified oases. 


Often  (particularly  in  business  and  economic  applications)  analysis  of 
relatively  short  time  series,  in  the  neighborhood  of  50-100  observations,  is 
required.  He  examine,  by  simulation,  the  behavior  of  various  procedures  for 
such  sample  sizes. 


Specifically,  we  examine  the  properties  of  three  estimators  of  the  coeffi- 
cients of  autoregressive-moving  average  models,  two  procedures  for  testing  the 
adequacy  of  representation  of  such  models  and  the  usual  estimates  of  error 
variances  when  these  models  are  projected  ahead  for  forecasting. 


I.  INTRODUCTION.  Suppose  that  the  available  data  consists  of  n obser- 
vations X]^,X2, . . . ,Xj^  on  a stationary  time  series.  (In  practice  it  is  often 
necessary  to  difference  the  original  data  to  induce  stationarity) . Such  data 
can  generally  be  well  represented  by  a low  order  autoregressive-moving  average, 
ARMA(p,q),  model 
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where  B is  a bac)(-shlft  operator  on  the  index  of  the  time  series  defined  so 
that  BJXt  B Xt-j,  and  at  is  white  noise,  i.e.,  E(at>  ■ 0,  B(a|)  - for 

all  t and  E(ataB)  ■■  0 for  all  t ^ s.  Stationarity  of  the  model  (1)  Is 
guaranteed  by  requiring  that  the  roots  of  the  polynomial  equation  in  B, 

(1  - ^jB  - ...  - ^pB^)  ■ 0,  all  have  modulus  greater  than  unity.  It  is  also 
convenient  to  impose  the  invertibility  condition,  that  the  roots  of 
(1  - Bj^B  - ...  - BqB*^)  ■ 0 all  have  modulus  greater  than  unity.  This  ensures 
uniqueness  of  representation  of  the  model.  A constant  term  can  be  added  to  (1) 
to  account  for  non-zero  series  mean. 


For  seasonal  time  series  of  period  s (so  that  s ■ 4 for  quarterly  and 
12  for  monthly  data)  the  model  (1)  can  be  elaborated  to  give  the  multiplicative 
seasonal  model 
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Stationarity  and  Invertlblllty  oondltlona  for  thla  model  are  obvious  extensions 
of  those  for  (1). 

Following  the  prlnoiplea  set  out  In  (1),  the  fitting  of  models  of  the  form 
(1)  or  (2)  to  data  Involves  an  iterative  cycle  of  identification,  estimation 
and  diagnostic  checking.  At  the  identification  stage,  based  on  statistics  cal- 
culated from  the  data,  a particular  modal  is  saleoted,  that  is  speoific  values 
for  p,Pg,q  and  g^  in  (2)  are  chosen.  The  parameters  of  this  model  are  then 
estimated  by  asymptotically  efficient  statistical  methods.  Finally  checks  are 
made  on  the  adequacy  of  representation  of  the  chosen  model  to  the  given  data. 

Any  inadequacies  revealed  at  this  stage  may  suggest  an  alternative  specification, 
and  the  cycle  is  iterated  until  a satisfactory  model  is  found.  The  model  even- 
tually obtained  may  than  be  projected  forward  to  obtain  forecasts  of  future 
values  of  the  time  series. 

Details  of  the  model  building  and  forecasting  methodologies  are  given  in 
the  books  (1),  (2}  and  (3)  and  more  briefly  in  the  paper  (4).  In  the  remainder 
of  this  section  we  describe  only  those  procedures  whose  properties  are  investi- 
gated in  the  present  study. 

Consider,  first,  the  problem  of  estimating  the  vector  § of  unknown  para- 
meters in  (1)  or  (2),  under  the  additional  assuii\ptlon  that  a^  is  normally 
distributed.  The  likelihood  function  can  then  be  written 

f(§)axp.[-S(§,J{^)/2oJ]  (3) 

where  ••  (X]^,X2, . . . ,Xf^)  and  f(g)  involves  the  parameters  but  not  the  data. 
Analytic  expressions  for  f(g)  and  S($,Xti)  are  given  in  (5),  and  an  alterna- 
tive form  which  can  lead  to  great  computational  savings  is  given  in  (6) . Maxi- 
mum likelihood  (M.L.)  estimates  of  g are  then  obtained  as  those  values  which 
maximise 

LjBlx  ) - f{g)(S(g,X„)]"‘'^^  (4) 
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Now,  clearly,  as  sample  size  increases  (4)  is  dominated  by  its  final  term.  If 
f(g)  is  treated  as  roughly  constant,  then,  this  suggests  use  of  the  exact  least 
squares  (E.Ii.S.)  estimator  which  involves  minimizing  S(§,X,i).  This  has  compu- 
tational advantages  over  M.L. , particularly  if  one  adopts  the  approximation 
involving  "back- forecasting"  proposed  in  (1).  The  name  "exact  least  squares" 
derives  from  the  fact  that  S(g,Xn)  can  be  written  as  a sum  of  squares. 

An  estimator  of  greater  computational  simplicity  is  obtained  by  writing, 
for  example,  (1)  as 
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If  i»3f*rq)  sre  set  to  their  expected  values,  zero, 

Sp+j 0 ■ l,...,n  - p)  can  then  be  calculated  recursively  from  (5)  as  functions 
of  the  and  Sj,  and  these  parameters  estimated  by  minimizing  the  sum  of 

r 2 

squares  I a . This  is  the  conditional  least  squares  (C.L.S.)  estimator. 
t-p+1  ^ 
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All  three  eetlmatora  are  calculated  by  numerical  function  minimization  and 
for  very  large  sample  sizes  they  are  virtually  Identical.  However,  In  small 
samples  there  are  li^portant  differences  and  these  are  Investigated  In  the  next 
section  of  this  paper. 

Now  let  I denote  the  coefficient  estimates  and  a^  the  residuals  from  the 
fitted  model.  Since,  If  the  model  is  oorreotly  specified,  the  a^  should  be 
white  noise,  It  is  natural  In  assessing  model  adequacy  to  examine  the  residual 
autocorrelations . 


t'w  ■ I ^ (6) 
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These  guantltlas  are  studied  in  (7)  and  (8),  where  it  la  shown  that,  If  the  modal 
la  oorreotly  specified,  they  are  asyirptotloally  normally  distributed  with  zero 
means  and  variances  which  we  write  as  V)t(§).  Unfortunately,  V)((3)  is  unknown, 
but  can  be  estimated  by  V]((@),  so  that  the  distribution  of 

S " “ If 2....)  (7) 

should  be  close  to  a standard  normal.  Clearly,  If  this  distributional  approxima- 
tion Is  valid,  then  large  absolute  values  of  the  statistics  (7)  will  indicate 
probable  model  Inadequacy. 

Also  In  (7)  a "portmanteau  test”  of  modal  inadequacy  Involving  the  first  m 
residual  autocurrelatlons  la  proposed.  It  J'  shown  that,  if  the  model  is  oor- 
reotly specified  the  statistic 

m , 
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Is  asymptotically  distributed  as  ohl-aquared  with  (m  - k)  degress  of  freedom, 
where  k lo  the  number  of  estimated  coefficients,  provided  m Is  sufficiently 
large  (values  of  m ^ 20  are  commonly  used) . 

In  fact  the  available  evidence  (see  (9)  and  (10))  suggests  that  In  finite 
samples  a closer  approximation  to  the  asymptotic  distributions  la  likely  to  be 
obtained  by  replacing  In  (7)  and  (8)  by 

r*  - [(n  + 2)/(n  - k>]^''^r,^  (k-1,2,...)  (9) 

giving  the  statistics  tj^  and  Q* . In  section  3 of  this  paper  we  examine  the 
empirical  distributions  or  these  statistics. 


Suppose  now  that  the  coefficients  of  (1)  or  (2)  are  known.  Betting 
t ■>  n + h,  these  equations  may  be  written  In  the  form 
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wh«r«  the  ijij  are  known  functions  of  the  and  6j.  It  is  then  wall  known 
that,  given  2'“''  minimum  mean  squared  error  predictor  of  Xn+h 

linear  In  X^-jCj  ■ Is  the  second  bracketed  expression  on  the  R.H.S. 

of  (10) . The  other  bracketed  term  is  then  the  forecast  error,  so  that  the  opti- 
mum predictor  has  error  variance 


V(h)  - aj(l  + I <1*?)  (11) 
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Now,  in  praotioe  there  are  two  reasons  why  (11)  oonstltutes  an  understate- 
ment of  the  best  attainable  foraoast  error  variance.  First,  even  If  the  model 
parameters  were  known,  given  only  a finite  realisation  of  the  time  series  the 
a„_j(j  • 0,1,2,...)  required  to  oonipute  the  optimal  predictor  would  be  unknown 
and  would  have  to  be  estimated  from  data.  Second,  the  modal  parameters  them- 
selves have  to  be  astlmatad,  and  sampling  variability  In  the  parameter  estimates 
naturally  leads  to  an  Increase  In  forecast  srror  variance  (see,  for  example,  (ID). 
Ihus  the  beat  attainable  V(h)  in  fact  depends  on  the  method  of  parameter  esti- 
mation employed,  and  oomperlsons  will  be  made  amongst  estimators  In  this  way  In 
the  following  section.  More  details  on  these  points  are  given  In  (12) . 

In  practice  forecast  error  variance  is  estimated  by  substituting  estimates 
of  oj  and  of  the  model  coefficients  to  oalculata  estimates  of  the  |^n  (II) . 
Ohls  Implies  the  possibility  of  further  bias  In  the  resulting  estimator  V(h) 
and  the  quality  of  this  as  an  estimator  of  forecast  error  variance  Is  examined 
In  section  4 of  thi^i  paper. 

The  results  presented  In  the  remainder  of  this  paper  summarise  an  extensive 
simulation  study  covering  a wide  range  of  non-saasonal  and  seasonal  time  series 
models.  While  we  will  discuss  our  findings  in  general,  specific  results  will  be 
quoted  for  just  two  models.  These  are  the  arma(1,1)  model 
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and  the  first  order  multiplicative  moving  average  quarterly  seasonal  model 

- (1  - 9^B) (1  - e^B4)a^  (13) 

More  detailed  results  are  contained  In  (12),  (13)  and  (14). 

II.  COMPARISON  OF  BSTIMATORS.  In  evaluating  the  performances  of  the  M.L., 
E.L.S.  and  C.L.S.  estimators  we  generated  data  from  a range  of  models  of  the 
form  (1)  and  (2),  taking  the  a^  to  be  standard  normal  deviates.  The  estimates 
were  compared  In  terms  of  bias,  mean  squared  error  and  the  quality  of  the  fore- 
casts resulting  from  their  use.  In  general  our  finding  wan  that.  If  a single 
estimator  Is  to  bo  recommendad  for  all-purpose  use,  M.L.  Is  preferable,  as  there 
are  olrcmistanoes  where  each  of  the  others  han  undesirable  features.  It  Is  not 
the  case  that  M.L.  Is  Invariably  best  by  any  of  our  criteria,  but  It  Is  rarely 
out-performed  to  any  great  extent. 

To  Illustrate,  Tables  1 and  2 contain  results  for  the  ARMA(1,1)  modal  (12) 
with  50  observations.  Clearly  for  this  sample  else  the  C.L.S.  estimator  can  be 
very  badly  biased  with  unacceptably  large  mean  squared  errors  for  large  values 
of  |6|  or  when  ^ and  0 are  close  to  one  another  In  value.  On  the  other 
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hand,  the  B.L.S.  estimator  oompares  rather  wall  with  M.L.  In  terms  of  bias  and 
moan  squared  error.  However,  use  of  this  estimator  rather  than  M.Tj.  would  lead 
to  slightly  inferior  performance  in  terms  of  forecast  error  variance,  except  in 
the  extreme  case  1 6 1 - 1 . 

Tables  3 and  4 contain  similar  results  for  the  quarterly  seasonal  model  (13) . 
The  performance  of  the  C.L.S.  estimator  for  this  model  is  rather  poor  for  moder- 
ate and  large  values  of  I64I,  Where  it  is  badly  biased  towards  sero.  On  the 
other  hand  B.L.S.  is  badly  biased  away  from  sero  for  small  and  moderately  large 
values  of  |94|.  Iheae  characteristios  are  reflected  in  poor  forecast  perfor- 
manoes  of  these  two  estimators.  Indeed,  by  this  criterion,  M.j;,,  seems  clearly 
preferable  except  in  extreme  cases  where  one  or  other  of  the  parameters  is  on 
the  boundary  of  the  invertlbility  region.  Here  B.l.S.  shows  up  rather  well. 
However,  its  doing  so  is  in  faot  a reflaotion  of  a very  undesirable  oharactar- 
istic  of  this  estimator.  Even  when  the  true  values  are  inside  the  boundaries  of 
the  parameter  region,  B.L.S.  estimators  are  quite  liltely  to  fall  on  these  bound- 
aries. This  point  is  illustrated  for  our  two  models  in  Tables  5 and  6. 

Clearly,  as  would  be  expected,  the  problem  is  more  severe  for  a sample  of 
50  observations  than  for  one  of  100  observations.  Nevertheless  it  is  by  no 
means  negligible  for  the  larger  sample  sisc.  The  importance  of  the  problem  lies 
in  the  fact  that,  if  the  B.L.S.  estimator  were  used,  the  analyst  could  frequently 
be  led  to  erroneous  oonclusions  about  the  appropriate  degree  of  differencing  for 
the  data. 

The  results  presented  in  this  section  are  a small  subset  of  these  contained 
in  (13).  From  this  larger  study  it  emerged  that  the  greatest  differences  between 
the  estimators  arise  in  models  with  moving  average  terms,  in  that  case  there  can 
be  problems  in  small  samples  with  the  use  of  either  of  the  least  squares  estima- 
tors. If  the  true  parameter  values  are  not  quite  far  from  the  boundary  of  the 
invertibility  region,  C.L.S.  estimates  can  be  badly  biased  to  the  center  of  that 
region,  with  large  mean  squared  errors  and  an  associated  poor  forecasting  per- 
formance. On  the  other  hand,  when  the  true  parameter  values  are  some  distance 
from  the  boundary,  the  B.L.S.  estimates  can  be  biased  towards  the  boundary  with 
rather  large  mean  squared  errors,  resulting  on  occasions  in  poor  forecasts. 
Moreover,  this  estimator  has  a disturbing  tendency  to  produce  estimates  on  the 
boundary  of  the  invertibility  region,  even  when  the  true  parameter  valuec  are 
well  inside. 

III.  STATISTICS  BASBD  ON  RESIDUAL  AUTOCORHBLATIONS.  We  assume  now  that  a 
time  series  model  has  been  fitted  to  data,  the  coefficients  having  bean  estimated 
by  maximum  likelihood.  In  checking  model  adequacy  it  is  natural  to  look  at  the 
residual  autocorrelations  for  the  first  one  or  two  lags  and  at  low  multiples  of 
the  seasonal  period.  The  statistics  tj^,  based  on  (7)  and  (9)  could  then  be 
used  to  suggest  possible  model  inadequacies.  However,  it  la  first  necessary'  to 
inquire  whether,  for  correctly  specified  models,  the  distribution  of  these 
statistics  is  sufficiently  close  to  standard  normal.  In  (14)  it  is  shown  that, 
for  first  order  autoregressive  and  moving  average  models,  even  for  samples  of 
only  so  observations,  the  distributional  agreement  in  the  tail  areas  is  quite 
close.  However,  for  two  parameter  models  the  situation  is  rather  less  clear,  as 
can  be  seen  from  Table  7. 

For  the  ARMA(1,1)  model  the  empirical  significance  levels  agree  very  well 
with  the  asymptotic  levels  for  k > 1,  even  for  sample  sise  50.  However,  for 
this  sample  size,  a test  based  on  the  first  residual  autocorrelation  would 
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rajaot  th«  hypothsils  of  correct  epeolficatlon  too  frequently  when  the  true  para- 
meter value*  are  fairly  small.  The  situation  Improves  markedly,  however,  ae  the 
sample  size  Increases  to  100.  The  only  case  which  gives  difficulty  hors  Is  the 
model 

\ ■ °*®Vi  “ % - ^•'‘•t-i 

which  again  would  be  rejected  far  too  often  by  a test  based  on  the  first  resid- 
ual autocorrelation,  in  faot  this  1*  not  alarming  ae,  for  such  a small  sample 
size,  lb  would  rarely  be  fitted  sinoe  In  praotlce  it  would  be  virtually  indis- 
tinguishable from  the  simpler  hypothesis  that  ■ et,  l.e.  that  the  data  is 
white  noise. 

This  is  typical  of  tha  results  found  In  (14)  for  non-seasonal  tnodels.  The 
test  statlatios  behavo  very  much  like  the  corresponding  asymptotic  distributions 
except  in  oases  of  near-overparameterlsatlon  (Whloh  would  be  unlikely  to  be  iden- 
tified in  practice) . In  these  circumstanoas  the  test  statistic  based  on  the 
first  residual  autocorrelation  tends  to  give  too  many  large  values. 

For  seasonal  time  series  models  it  is  natural  to  check  not  only  tha  low  order 
residual  autocorrelations,  but  also  those  at  small  multiples  of  tha  seasonal 
period.  For  the  quarterly  moving  average  model  (13)  seme  results  are  shown  in 
Table  6.  Of  course,  it  is  almost  invariably  tha  oasa  that  tha  agreement  between 
empix'iael  end  asymptetic  distributions  improves  as  sample  size  increases.  Never- 
theless, the  improvement  from  sample  size  50  to  100  her*  is  remarkable.  For  t)ie 
larger  sample  size  the  empirical  significance  levels  are  generally  quite  close 
to  the  asymptotic  levels,  suggesting  that  for  this  sample  size  Interpretation  of 
the  statistics  is  straightforward.  On  tha  other  hand,  for  tha  smaller  sample 
size  the  empirical  signlfloanoo  levels  are  frequently  too  high,  particularly  at 
those  lags  assneisted  with  the  seasonal  frequency.  These  results  are  typical  of 
these  reported  in  (14) . 

We  now  consider  the  portmanteau  statistic  Q*,  based  on  (8)  and  (9). 

Although  this  statistic  is  almost  invariably  calculated  in  practical  time  series 
studies,  published  evidence  of  its  empirical  distribution  is  sparse,  the  prime 
exception  being  in  (10) , where  just  the  simple  first  order  autoregreasivet  modal 
is  examined.  Tables  9 and  10  show  empirical  significance  levels  for  the  models 
(12)  and  (13).  The  evidence  in  these  tables  certainly  reflects  variability 
between  models  and  also  variability  between  parameter  values  within  the  same 
model.  However,  it  is  quite  clear  (and  this  is  confirmed  by  further  evidence  in 

(14) }  that  tha  empirical  significance  levels  are  generally  "too  high".  This 
observation  accords  with  a prediotlon  made  from  theoretical  considerations  in 
(9) . Generally  speaking,  the  diacrepancles  between  the  empirical  and  asymptotic 
distributions  are  likely  to  be  largest  for  small  sample  sizes,  for  seasonal 
models,  and  in  the  extrema  tall  areas  of  the  distributions.  However,  it  is  clear 
frcM  the  tables  that  even  for  samples  of  100  observations,  for  such  relatively 
simple  two  parameter  models,  use  of  the  Q*  statistic  can  lead  to  rejection  of 

a correctly  specified  model  far  more  often  than  reference  to  the  asymptotic  sig- 
nificance levels  vrould  indicate. 

An  additional  consideration  when  using  any  test  statistic,  of  course,  con- 
cerns its  power.  Some  evidence  on  the  frequency  with  which  the  portmanteau  test 
detnets  model  misspeoiflaationB  of  varying  degrees  of  severity  is  contained  in 

(15) ,  where  it  was  found  that,  unless  the  sample  size  is  large,  the  test  can 
have  disturbingly  low  power. 
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IV.  ESTIMATION  OF  FORECAST  ERROR  VARIANCE.  When  a fitted  time  series  model 
is  projeoted  forward  to  obtain  forecasts,  it  is  usual  to  estimate  the  error  vari- 
ance by  substituting  estimates  of  the  unknown  parameters  in  (11) . The  i|/j  in 
that  expression  are  estimated  in  an  obvious  way  from  the  coeffiolent  estimates 
and  3j.  If,  as  will  be  assumed  in  this  suction,  the  model  is  estimated  by 
maximum  likelihood,  it  follows  from  (3)  that  the  Innovation  variance  Og  is 
estimated  by 

al  - S(8,x^)/n  (15) 

a ~ "“n 

where  | Is  the  vector  of  maximum  likelihood  coefficient  asjfimates.  With  these 
substitutions,  we  denote  the  estimate  of  V(h)  of  (11)  as  V(h). 

In  fact,  as  we  have  already  noted,  the  minimum  attainable  forecast  error 
variance,  V„(h),  will  be  g-:aatar  than  V(h).  This  is  so,  since  the  expression 
(11)  takes  no  account  of  sampling  variability  in  the  parameter  estimates  or  of 
the  fact  that  only  the  finite  past  of  a time  series  is  avaj.lable  for  the  compu- 
tation of  forecasts.  It  is  likely  then  that  V(h)  will  be  a biased  estimator 
of  V^(h).  In  this  section  we  examine  the  extent  of  that  bias.  It  should  be 
emphasised  that  our  results  are  specific  to  the  case  where  parameter  estimation 
is  by  maximum  likelihood.  In  particular  clrcumstanoas,  rather  different  conolu- 
slons  can  hold  for  alternative  estimators,  as  illustrated  in  (12). 

Tables  11  and  12  contain  values  of  (E(V(h))  - V^(h))/v^(h)  estimated  by 
simulation  for  the  models  (12)  and  (13).  The  general  picture  emerging  from  these 
tables  la  of  a moderate  downward  bias  in  \^(h) . For  one  step  ahead  prediction  In 
the  ARMA(1,1)  model  this  is  in  the  neighborhood  of  5-10%  of  the  true  variance  for 
sample  else  50,  and  3-6%  for  samples  of  100  observations.  For  the  seasonal  model 
the  oorresponding  figures  are  slightly  higher.  Perhaps  the  outstanding  feature 
of  the  tables  concerns  prediction  10  steps  ahead  for  ARMA(1,1)  models  with  the 
higher  autoregressive  parameter  value.  Here  the  bias  can  be  around  20%  of  the 
true  variance  for  50  observations  and  around  12%  for  twice  that  sample  sise. 

Although,  with  this  latter  exception,  the  tables  suggest  some  uniformity  in 
the  proportionate  bias,  the  causes  of  that  bias  differ  substantially  between 
models  and  parameter  values.  We  suggested  previously  four  potential  causes  of 
bias  in  the  usual  estimator  of  forecast  error  variance.  It  is  of  interest,  now, 
to  examine  these  factors  in  a little  more  detail. 

Suppose,  for  now,  that  the  parameter  values  are  given.  It  is  still  the  case 
that,  if  the  model  contains  moving  average  terms  and  only  the  finite  past  is 
available  to  compute  forecasts,  the  minimum  attainable  forecast  error  variance 
will  be  somewhat  higher  than  (11).  This  factor  is  only  of  practical  aignifioance 
whan  the  moving  average  coefficients  are  on  or  very  close  to  the  boundary  of  the 
invertibility  region.  Even  then,  for  the  kind  of  sample  slses  considered  here, 
it  is  relatively  unimportant  for  simple  non-seasonal  models.  For  exampla,  for 
the  ARMA(1,1)  models  of  Table  11  for  sample  sise  50  it  accounts  for  an  addition 
of  at  most  2%  (when  6 ••  -1)  to  one  step  ahead  prediction  error  variance,  and 
less  for  prediction  further  ahead.  On  the  other  hand,  for  seasonal  moving  aver- 
age models,  in  small  samples  this  factor  can  be  more  substantial.  For  example, 
for  the  multiplicative  first  order  moving  average  quarterly  model,  in  the 
extreme  case  6^^  ■ 1,  64  ■ 1,  the  minimum  attainable  one-step  forecast  error 
variance  is  13.4%  higher  than  (11)  for  sample  else  50  and  6.9%  higher  for  sample 
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size  100.  However,  the  effect  quickly  dies  out  as  the  pararaetere  move  away  from 
the  boundary.  For  exan\ple,  for  ■ 0.85,  0^  ■ 0.85  the  inflation  factor  for 
one  step  ahead  prediction  is  only  0.5%  for  sample  size  50. 


The  necessity  to  estimate  the  model  parametare  Inflates  forecast  error  vari- 
ance by  a proportion  in  the  neighborhood  of  K/n,  where  K is  the  nunibar  of 
estimated  coefficients.  There  is,  of  course,  some  variability  here.  A particu- 
lar case  is  in  forecasting  several  steps  ahead  when  the  optimal  predictor  la 
dominated  by  a relatively  low  autoregressive  term.  In  this  case  the  influence 
of  estimation  error  can  be  very  slight.  For  example,  for  the  ARMA(1,1)  modal 
for  i|i  ■■  0.5,  6 ■ -0.4,  h " 10,  estimation  error  in  the  parameters  adds  only 
0.1%  to  (11).  This  factor  is  discussed  in  more  detail  for  non-seasonal  models 
in  (11)  and  (16). 

For  maximum  likelihood  estimation,  the  estimate  (15)  of  rasidual  variance 
tends  to  be  slightly  biased  downwards.  An  exception  is  the  case  where  moving 
average  coefficients  are  on  or  very  close  to  the  boundary  of  the  invertibility 
region,  when  the  bias  is  upwards.  In  this  latter  case  some  of  the  effects  of 
the  inflation  caused  by  only  having  the  finite  past  to  calculate  forecasts  are 
cancelled  out,  and  this  explains  why  the  estimates  of  error  variance  when  moving 
average  terms  are  on  the  boundary  of  the  invertibility  region  do  not  have  a more 
severe  downward  bias.  For  example,  for  the  multiplicative  first  order  moving 
average  quarterly  model  with  o*  ■ i n ■ 50,  for  « 0,85,  64  ■ 0.85, 

B(o|)  " 0.92,  while  for  ■ 1,  84  ■ 1,  E(a2)  - 1,05. 

2 

Finally,  the  term  I in  (11)  is  generally  well  estimated  by  substitu- 
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tion  of  the  parameter  estimates,  except  for  moderately  large  h in  models  con- 
taining autoregressive  factors  close  to  the  stationarity  boundary.  This  is  the 
reason  for  the  serious  under-estimation  of  forecast  error  variance  in  the 
ABMA(1,1)  model  for  ^ ••  0.95,  h ■ 10  noted  in  Table  11, 

These  factors  are  all  discussed  in  mors  detail  for  a wider  range  of  models 
in  (12). 
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Table  li  Estimated  coefficient  biases  for  ARMA(1,1)  process 
[n  - 50;  1,000  replications] 

Bias  i Bias  i 


t 

9. 

M.I.. 

Eela*  S e 

Cali*  S a 

M.I.. 

Ee  lie  Se 

C.L.S. 

0.95 

-1.00 

0.172 

0.096 

0.505 

0.112 

0.042 

0.468 

0.95 

-0.85 

0.090 

0.067 

0.298 

0.074 

0.050 

0.286 

0.95 

-0.40 

0.039 

0.017 

0.046 

0,026 

0.006* 

0.029 

0.95 

0.40 

0.028 

0.012 

0.033 

0.015 

0.033 

0.017 

'0.95 

0.85 

0.027 

0.009 

0.024 

0.025 

0.053 

-0.012 

0.95 

1.00 

0.026 

0.007 

0.023 

-0.028 

-0.008 

-0.098 

0.50 

-1.00 

0.064 

0.054 

0.184 

0.041 

0.011 

0.180 

■0.50 

-0.85 

0.061 

0.053 

0.191 

0.001* 

-0.017 

0.143 

-0.50 

-0.40 

0.119 

0.064 

0.191 

0.114 

0.047 

0.178 

-0.50 

0.40 

0.038 

0.031 

0.016 

0.032 

0.040 

0.018 

•0.50 

0.85 

0.007* 

0.004* 

-0.001* 

0.025 

0.056 

-0.016 

-0,50 

1.00 

0.005* 

-0.008 

-0.026 

-0.032 

-0.008 

-0.105 

0.50 

-1.00 

-0.013 

0.006* 

0.010 

0.027 

0.007 

0.105 

0.50 

-0.85 

-0.020 

-0.010 

0.007* 

-0.021 

-0.057 

0.018 

0.50 

-0.40 

-0.033 

-0.020 

-0.016 

-0.036 

-0.034 

-0.013 

0.50 

0.40 

-0.120 

-0.064 

-0,18’ 

-0.114 

-0.056 

-0.164 

0.50 

0.85 

-0.072 

-0.046 

-0.183 

-0.009* 

0.025 

-0.128 

0.50 

1.00 

-0.064 

-0.055 

-0.182 

-0.040 

-0.011 

-0.181 

0.95 

-1.00 

-0.028 

-0.004 

-0.024 

0.030 

0.008 

0.105 

0.95 

-0.85 

-0.026 

-0.010 

-0.025 

-0.020 

-0.056 

0.019 

0.95 

-0.40 

-0.030 

-0.007 

-0.035 

-0.01" 

-0.028 

-0.018 

0.95 

0.40 

-0.046 

-0.017 

-0.047 

-0.03 

-0.012* 

-0.015* 

0.95 

U.85 

-0.096 

-0.050 

-0.281 

-0.07' 

-0.028 

-0.264 

0.95 

1.00 

-0.168 

-0.103 

-0.526 

-O.lli: 

-0.051 

-0.500 

*Biaa  is  not  statistically  significant  at  5%  le\;el. 
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Table  2;  Estimated  coefficient  mean 
for  ASMA(1,1)  process  [n  ■ 


squared  errors  and  forecast  error  variances 
50)  1,000  replications] 


M.S 

.E.  $ (x 

103) 

M.S. 

E.  0 (X  103) 

V(l) 

1 

e 

M.L. 

B a Le  S s 

C«  Lie  S • 

H.L. 

Be  Li*  S • 

C.L.S. 

M.L. 

H aLl*  S • 

C * Lie  S • 

0.95 

-1.00 

90.5 

28.1 

502 

63.8 

23.2 

510 

1.054 

1.050 

1.065 

0.95 

-0.85 

52.0 

50.7 

298 

70.8 

86.2 

320 

1.036 

1.058 

1.057 

0.95 

-0.40 

9.41 

7.09 

13.2 

33.4 

34.3 

36.8 

1.037 

1,053 

1.048 

0.95 

0.40 

4.78 

4.85 

5.54 

22.7 

30.2 

23.6 

1.043 

1.051 

1.049 

0.95 

0.85 

4.40 

3.86 

4.36 

9.04 

12.5 

10,1 

1.039 

1.063 

1.047 

0.95 

1.00 

3.91 

3.20 

5.19 

3.11 

1.12 

17.8 

1.055 

1.050 

1.101 

0.50 

-1.00 

29.0 

20.9 

91.7 

9.60 

2.09 

104 

1.059 

1.053 

1.114 

■0.50 

-0.85 

57.6 

59.2 

122 

38.6 

40.1 

121 

1.041 

1.059 

1.053 

0.50 

-0.40 

234 

214 

283 

287 

255 

323 

1.044 

1.047 

1.045 

■0.50 

0.40 

34.1 

35.4 

29.5 

40.8 

43.0 

40.3 

1.049 

1.049 

1.050 

■0.50 

0.85 

17.1 

17.1 

18.1 

11.1 

12.9 

12.6 

1.047 

1.061 

1.054 

■0.50 

1.00 

14.4 

14.7 

18.4 

4.18 

0.95 

20.0 

1.066 

1.047 

1.100 

0.50 

-1.00 

13.8 

13.8 

20.1 

3.46 

0.82 

20.8 

1.055 

1.046 

1.095 

0.50 

-0.85 

19.5 

17.6 

17.8 

10.5 

13.7 

12.5 

1.045 

1.068 

1.058 

0.50 

-0.40 

31.9 

31.3 

30.0 

40.8 

45.0 

42.9 

1.046 

1.053 

1.051 

0.50 

0.40 

241 

199 

282 

294 

253 

329 

1.043 

1.044 

1.046 

0.50 

0.85 

60.9 

54.8 

118 

40.4 

38.2 

104 

1.046 

1.062 

1.059 

0.50 

1.00 

30.1 

24.8 

90.3 

9.36 

4.76 

96.9 

1.059 

1.054 

1.104 

0.95 

-1.00 

3.71 

3.08 

4.08 

3.72 

1.10 

20.3 

1.056 

1.049 

1.101 

0.95 

-0.85 

3.64 

4.06 

4.76 

10.1 

12.6 

11.6 

1.039 

1.065 

1.046 

0.95 

-0.40 

5.16 

4.12 

6.02 

23.8 

23.4 

23.9 

1.039 

1.051 

1.044 

0.95 

0.40 

14.0 

9.91 

13.2 

38.4 

38.9 

38.6 

1.042 

1.060 

1.054 

0.95 

0.85 

67.4 

39.9 

258 

87.6 

71.7 

276 

1.041 

1.070 

1.051 

0.95 

1.00 

93.9 

42.4 

520 

68.6 

36.6 

547 

1.048 

1.050 

1.060 

[ 
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Table  3:  Estimated  coefficient  biases  for  multiplicative  first  order 
moving  average  quarterly  models  [n  » 50;  600  replications] 

Bias  §1  Bias  §4 


M.L. 

Bella  Se 

C.L.S. 

M.L. 

E.L.S. 

C a Xf  • S a 

-1.00 

-1.00 

0.030 

0.011 

0.085 

0.083 

0* 

0.257 

-1.00 

-0.85 

0.035 

0.010 

0.089 

-0.020 

-0.133 

0.128 

-1.00 

-0.40 

0.031 

0.009 

0.091 

0.005* 

-0.056 

0.010* 

-1.00 

0.40 

0.039 

0.006 

0.171 

0.046 

0.149 

-0.006 

-1.00 

0.85 

0.048 

0.004 

0.256 

0.033 

0.142 

-0.164 

-1.00 

1.00 

0.042 

0.001 

0.264 

-0.064 

0* 

-0.287 

-0.85 

-1.00 

-0.012 

-0.038 

0.004* 

0.082 

0* 

0.245 

-0.85 

-0.85 

-0.018 

-0.045 

0.013 

-0.014 

-0.132 

0.122 

-0.85 

-0.40 

-0.022 

-0.049 

0.005* 

-0.007* 

-0.075 

0.007* 

-0.85 

0.40 

-0.018 

-0.054 

0.062 

0.030 

0.105 

0.000* 

-0.85 

0.85 

-0.025 

-0.094 

0.141 

0.010* 

0.141 

-0.156 

-0.85 

1.00 

-0.029 

-0.103 

0.147 

-0.074 

0* 

-0.278 

-0.40 

-1.00 

-0.006* 

-0.010* 

-0.010* 

0.076 

0* 

0.241 

-0.40 

-0.85 

-0.003* 

-0.013 

0.002* 

-0.022 

-0.130 

0.117 

-0.40 

-0.40 

-0.017 

-0.015 

-0.014 

-0.008* 

-0.079 

-0.007* 

-0.40 

0.40 

0.003* 

-0.025 

-0.001* 

0.017 

0.084 

-0.009* 

-0.40 

0.85 

-0.019 

-0.044 

0.005* 

0.007* 

0.137 

-0.119 

-0.40 

1.00 

-0.007* 

-0.032 

0.007* 

-0.080 

0 

-0.249 

*Blas  Is  not  statistically  significant  at  5%  level. 
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Table  4t  Estimated  coefficient  mean  squared  errors  and  forecast  orror  variances 
for  multiplicative  first  order  quarterly  moving  average  models 
[n  ■ SO;  600  replications] 


M.S. 

E.  §1  (x  103) 

M.S. 

E.  §4  (x  103) 

M.L. 

V(l) 

C.L.S 

M.L. 

Ls  S s 

C.L.S. 

M.L. 

E • Li » S k 

Ce  Ls  S s 

Ee  Le  S f 

-1.00 

-1.00 

2.83 

1.46 

13.2 

17.4 

0 

80.1 

1.149 

1.113 

1.471 

-1.00 

-0.85 

4.30 

1.13 

13.7 

14.9 

22.4 

31.1 

1.077 

1.055 

1.250 

-1.00 

-0.40 

3.79 

1.09 

15.3 

23.0 

49.8 

23.8 

1.072 

1.089 

1.189 

-1.00 

0.40 

6.42 

1.09 

44.4 

29.6 

80.1 

26.9 

1.066 

1.133 

1.272 

-1.00 

0.85 

6.67 

0.92 

84.5 

14.1 

22.5 

41.1 

1.092 

1.070 

1.425 

-1.00 

1.00 

6.26 

0.24 

90.7 

13.5 

0 

95.7 

1.182 

1.134 

1.503 

-0.85 

-1.00 

7.79 

12.9 

9.59 

17.2 

0 

74.1 

1.126 

1.124 

1.410 

-0.85 

-0.85 

8.06 

12.0 

9.90 

16.3 

23.2 

28.8 

1.051 

1.065 

1.192 

-0.85 

-0.40 

9.39 

12.9 

9.38 

26.9 

53.6 

24.5 

1.051 

1.108 

1.140 

-0.85 

0.40 

11.3 

13.5 

16.1 

32.1 

63.1 

23.9 

1.052 

1.113 

1.076 

-0.85 

0.85 

11.8 

18.2 

39.3 

13.2 

22.6 

38.5 

1.057 

1.068 

1.178 

-0.85 

1.00 

9.19 

19.1 

42.9 

15.3 

0 

89.9 

1.130 

1.124 

1.383 

-0.40 

-1.00 

15.8 

20.0 

22.6 

17.0 

0 

70.0 

1.127 

1.102 

1.205 

-0.40 

-0.85 

18.9 

24.8 

21.3 

17.8 

22.5 

25.2 

1.047 

1.056 

1.087 

-0.40 

-0.40 

25.9 

24.3 

24.0 

27.9 

52.0 

23.6 

1.057 

1.088 

1.065 

-0.40 

0.40 

21.7 

25.5 

21.1 

26.6 

56.  3 

23.9 

1.045 

1.106 

1.048 

-0.40 

0.85 

22.7 

36.2 

26.7 

19.7 

22.7 

26.7 

1.052 

1.074 

1.092 

-0.40 

1.00 

19.5 

33.4 

25.3 

17.8 

0 

73.9 

1.128 

1.121 

1.218 
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Table  6 
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Percentage  times  an  E.L.S,  parameter  estimate  is  Of  the  boundary 
of  the  stationarity  or  Invertibility  region  for  ARMA(1,1)  process 
[1,000  replications,  n«S0|i  600  replications,  n>100] 


n“50 

n-100 

i 

£ 

n«50 

n-100 

85 

46.7 

8.7 

0.95 

-0.85 

47.4 

17.3 

40 

1.4 

0 

0.95 

-0.40 

10.8 

0.3 

40 

15.2 

5.5 

0.95 

0.40 

19.8 

0.8 

85 

53.0 

26.8 

0.95 

0.85 

54.0 

19,8 

Percentage  times  an  E.L.S.  parameter  estimate  is  on  the  boundary 
of  the  Inver clbility  region  for  multiplicative  first  order  moving 
average  quarterly  models  [600  replications] 


h. 

n-SO 

n-100 

n-50 

n-100 

1.40 

9.  J 

0.2 

0.85 

0.40 

57.3 

22.2 

1.85 

92.3 

73.7 

0.85 

0.85 

98.0 

78.3 

75 


Table  7;  Bn\pirlcal  significance  levels  of  the  statistic  tj^  for  ARMA(1,1) 
models  [1,000  replications,  n * 50t  600  replications,  n ■■  100] 


n - 

50 

n - 100 

1 

B 

K 

0.05  level 

0.10  level 

0.05  level  0 

.10  level 

0.50 

-0.85 

1 

0.111 

0.186 

0.072 

0.128 

2 

0.063 

0.109 

0.040 

0.095 

3 

0.052 

0.112 

0.055 

0.103 

0.50 

-0.40 

1 

0.184 

0.257 

0.068 

0.130 

2 

0.050 

0.105 

0.055 

0.100 

3 

0.045 

0.099 

0.057 

0.112 

0.50 

0.40 

1 

0.147 

0.195 

0.197 

0.248 

2 

0.055 

0.107 

0.095 

0.143 

3 

0.051 

0.103 

0.047 

0.097 

0.50 

0.85 

1 

0.072 

0.141 

0.058 

0.108 

2 

0.052 

0.112 

0.058 

0.098 

3 

0.049 

0.092 

0.042 

0.100 

0.95 

-0.85 

1 

0.066 

0.133 

0.053 

0.100 

2 

0.059 

0.099 

0.043 

0ill2 

3 

0.065 

0.110 

0.058 

0.110 

0.95 

-0.40 

1 

0.101 

0.157 

0.063 

0.120 

2 

0.065 

0.116 

0.077 

0.130 

3 

0.052 

0.114 

0.053 

0.083 

0.95 

0.40 

1 

0.113 

0.175 

0.078 

0.122 

2 

0.060 

0.102 

0.045 

0.093 

3 

0.041 

0.086 

0.062 

0.120 

0.95 

0.85 

1 

0.068 

0.130 

0.055 

0.130 

2 

0.058 

0.133 

0.062 

0.110 

3 

0.057 

0.109 

0.055 

0.090 

Table  8 1 Entplrloal  slgnlfloemoe  levela  of  the  etatiatic  for  multiplicative 

firet  order  moving  average  quarterly  modela  [600  replications] 


! 

! 


•V 
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n ■ SO  n ■ 100 


K 

0.05  level 

0.10  level 

0.05  level 

0.10  level 

0.40 

0.40 

1 

0.060 

0.100 

0.053 

0.102 

2 

0.040 

0.080 

0.038 

0.097 

4 

0.067 

0.125 

0.047 

0.102 

8 

0.057 

0.105 

0.045 

0.093 

0.40 

0.85 

1 

0.105 

0.152 

0.063 

0.133 

2 

0.0.50 

0.087 

0.055 

0.107 

4 

0.085 

0.150 

0.050 

0.107 

8 

0.085 

0.120 

0.073 

0.135 

0.85 

0.40 

1 

0.052 

0.100 

0.050 

0.093 

2 

0.048 

0.102 

0.047 

0.108 

4 

0.107 

0.163 

0.058 

0.122 

8 

0.072 

0.135 

0.052 

0.095 

0.85 

0.85 

1 

0.048 

0.108 

0.052 

0.125 

2 

0.042 

0.093 

0.050 

0.093 

4 

0.093 

0.162 

0.045 

0.087 

8 

0.097 

0.165 

0.057 

0.128 

i 


Table  9t  Bbiplrloal  algnlfloanoe  lavala  of  the  atatletlc  Q*  for  ABHAd^l)  models 
[m  ■>  20i  IfOOO  replioatlons,  n - 50)  600  repllaatlons,  n ■ 100] 


i 

0.05  level 

n - 50 

0.10  level 

0.20  level 

0.05  level 

n - 100 

0.10  level 

0.20  level 

o.so 

-0.85 

0.104 

0.169 

0.267 

0.053 

0.112 

0.212 

0.50 

-0.40 

0.069 

0.106 

0.198 

0.068 

0.110 

0.208 

0.50 

0.40 

0.080 

0.132 

0.224 

0.072 

0.123 

0.200 

O.SO 

0.85 

0.100 

0.152 

0.243 

0.068 

0.112 

0.212 

0.95 

-0.85 

0.102 

0.153 

0.267 

0.065 

0.132 

0.226 

0.95 

-0.40 

0.092 

0.146 

0.256 

0.072 

0.128 

0.208 

0.95 

0.40 

0.078 

0.123 

0.218 

0.077 

0.138 

0.248 

0.95 

0.85 

0.106 

0.179 

0.272 

0.105 

0.167 

0.275 

Table  lOi  Bngplriaal  elgnifiaanoe  levels  of  the  atetiatlc  Q*  for  multiplicative 
firat  order  moving  average  quarterly  models  [m  <■  24;  600  raplicatlona] 


li 

0.05  level 

n 50 

0.10  level 

0.20  level 

0.05  level 

n ■ 100 

0.10  level 

0.20  level 

0.40 

0.40 

0.065 

0.123 

0.210 

0,050 

0.100 

0.212 

0.40 

0.85 

0.135 

0.223 

0.330 

0.088 

0.157 

0.245 

0.85 

0.40 

0.105 

0.143 

0.253 

0.077 

0.133 

0.213 

0.85 

0.85 

0.168 

0.235 

0.338 

0.095 

0.160 

0.255 
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Table  111  EBtlmates  of  (E(V(h))  - Vj^(h))A,  (h)  for  ARMA(1,1)  models 
[1,000  replloatlons,  n-50|  600  replications,  n - 100] 


Table  12i  Estimates  of  (R(V(h))  - V^(h))/V*(h)  for  multiplicative 

first  order  moving  average  quarterly  models  [600  replications] 


REFERENCES 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 


Boxi  Q.E.P.  and  Janklni,  Q.M.  (1970).  Tima  Sarlas  Analvala,  Forapastinct  and 
Control . Holdan  Day,  San  Franolaoo. 


Nalaon,  C.R.  (1973).  Rppllad  Tima  sarlaa  Anaiyais 
Holdan  Day,  San  Franolaoo. 


for  Manaqarlal 


Foraoaatln^ 


Grangar,  C.W.J.  and  Kawbold,  P.  (1977).  Poraoaatlnq  Eoonottilo  Tima  Sarlaa. 
Aoadamlo  Praia,  Naw  Yor]t. 

Nawbold,  P.  (1975).  nia  prlnolplaa  of  tha  Box-Jan](lna  approach.  Oparatlonal 
Raaaaroh  Quartarly,  26,  397-412. 

Nawlrald,  P.  (1974).  Tha  axaot  lllcallhood  function  for  a mixed  autoregreaalva 
moving  avaraga  prooasa.  Blomatrllta,  61,  423-426. 

Analay,  C.F.  (1979).  An  algorithm  for  tha  exact  IDcelihood  of  an  auto- 
ragraaalva-moving  avaraga  proooas.  Blomatrllta.  66  (forthcoming) . 

Box,  Q.B.P.  and  Piaroa,  D.A.  (1970).  Dlatrlbutlon  of  raaidual  autooorra- 
lationa  In  autoragraaaiva  Intagratad  moving  avaraga  tlma  aariea  modala. 
Journal  of  Amarloan  Statlatioal  Aaeoeiation.  65,  1509-1526. 

MoLaod,  A.X.  (1977).  On  tha  dlatrlbutlon  and  applioatlona  of  raaidual  auto- 
corralatlona  in  Box-Janklni  modala.  Dapartmant  of  Statiatioa,  Univaralty  of 
Waterloo. 

Daviaa,  N.,  Trlgga,  C.N.  and  Nawbold,  P.  (1977).  Signifioanoa  levala  of  tha 
Box-Piarca  portmanteau  atatiatio  in  finite  aamplaa.  Biometrilca,  64,  517-522. 

Ljung,  G.M.  and  Box,  G.E.P.  (1978).  On  a measure  of  lack  of  fit  in  time 
sarlaa  models.  Biometrika,  65. 

Yamamoto,  T.  (1976).  Asymptotic  mean  square  prediction  error  for  an  auto- 
ragrosslva  model  with  estimated  coefficients.  Applied  Statistics,  25, 
123-127. 

Analay,  C.F.  and  Nawbold,  F.  (1979).  On  the  blaa  in  eatimatea  of  forecast 
error  variance.  Graduate  School  of  Business,  University  of  Chicago. 

Analay,  C.F.  and  NewlDold,  P.  (1979).  Finite  seunpla  properties  of  estimators 
for  autoragrasslva-moving  average  models.  Graduate  School  of  Business, 
university  of  Chicago. 

Ansley,  C.F.  and  Nawlsold,  P.  (1979).  On  the  finite  sample  distribution  of 
residual  autocorrelations  In  autoregresslve-movlng  average  models.  Graduate 
School  of  Business,  University  , of  Chicago. 

Davies,  N.  and  Newbold,  P.  (1979).  Some  power  studies  of  a portmanteau  test 
of  time  series  model  specification.  Biometrika,  66  (forthcoming) . 

Yamamoto,  T.  (1977^.  Prediction  error  of  parametric  time  series  models  with 
astlmatad  coefficients.  Department  of  Economics,  Soka  University,  Tokyo. 


80 


yin 


i ' 


STATISTICAL  PROBLEMS  ASSOCIATED  WITH  THE  HORIZONTAL  CHANNEL 
OF  THE  RAPID  GEODETIC  SURVEY  SYSTEM  (RGSS) 
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K.  Bausiua  von  Luattow 
U.S.  Azny  Bnglnaar  Topographie  LaJsoratoriaa 
Fort  Ml  voir  f Virginia 

ABSTRACT 

nia  papar  diaeusaaa  tha  aatimatlon  of  daflaotiona  of  tha  vartloal  along  with 
heriaental  gyro  blaaaa  from  a aat  of  glvsn  and  maaaurad  data  at  a atatiatioal 
adjuatmant  problam.  In  oonjunotlon  harawith,  it  praaanta  a guaal-optlmal  aati- 
aation  mathod  and  naoaaaary  eovarianoa  funetlona.  It  furthar  outllnaa  tha 
aatimatlon  of  grlddad  daflaotiona  from  ROSS  data  and  inprovad  atoohaatio 
poaition  arror  control. 

1.  INTRODUCTION 

Tha  involvamant  of  tha  U.S.  Army  Bnginaar  TopogrAphie  Laboratoriaa  (ETL)  in  tha 
fiald  of  inartlal  aurvaying  and,  aubaaquantly,  in  inartial  gaodaay,  can  ba 
oharaotariaad  by  thraa  phaaaa.  Phaaa  I oon\prlaad  tha  davalopmant  of  a Poaition 
and  Aaimuth  Datarmining  Syatam  (PADS)  primarily  in  aiypport  of  U.S.  Amv 
artillary  and  waa  ooiqplatad  in  19,72.  Phaaa  II  waa  eonoamad  with  the  Inatal- 
lation  of  a hlghar-aoouraey  vartloal  aoealaromatar  for  inprovad  vartloal 
poaitloning  and  tha  inoluaion  of  aoftwara  and  a data  atoraga  unit  for  tha 
datarmlnation  of  gravity  anomaliaa  and  daflaotiona  of  the  vartloal  oonponanta 
( and  n undar  oonaldaration  of  initial  and  taminal  gravity  vector  oonponanta. 

Tha  modified  PADS  oparataa  aa  an  optimal  looal-laval  ayatam  In  tha  Inertial 
Poaitloning  Syatam  (IPS)  mode  and  aa  a quaai-local-level  ayatam  in  tha  Rapid 
Oaodatio  Survey  Syatam  (ROSS)  mode.  Tha  RGSS  mode  without  Kalman  platform  tilt 
oorraotiona  haa  advantagaa  oonoamlng  gyro  biaa  aatimationa  and  thua  for  tha 
detaiminatlon  of  C and  n . Phaaa  XI  waa  aaaantlally  oomplatad  after  ETL  teata 
at  White  Sanda  Mlaaila  Range  In  1976.  Thaaa  teata  aatabliahed  an  RGSS  capability 
of  datarmining  gravity  anomaliaa  and  dafleotion  componanta  with  average  rma 
arrora  of  2 mgal  and  2 arcaeo,  reapactivaly  for  50  km  runa  in  ooaparlaon  with 
unreduced  rma  valuta  of  35  mgal  and  5 orcaac.  Phaaa  III  ooncentrataa  aaaantlally 
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on  additional  RGSS  testing  in  the  vicinity  of  Washington,  D.C.,  the  development 
of  iiqprovad  methods  for  the  determination  of  € and  n and  their  iiqplementation , 
intproved  atochaetio  error  control  for  positioning,  and  desirable  hardware 
improvements,  notably  the  installation  of  two  AlOOO  accelerometers  in  the 
horisontal  channel.  With  sufficient  funding,  it  could  be  coiqpleted  in  1981 
and  should  permit  i,  n-determinations  with  a mean  standard  error  between  0.5 
and  1.0  arooeo  without  repetitions,  and  improved  positioning.  Promising 
BOSS  applications  arei 

*Rapid  Ag,  n-determinations  along  solitary  courses  of  about  60  km  length. 

*Bstablishment  of  regional  Ag,  n~grid  information  networks  suitable  for 
use  in  a gravlty~programmed  inertial  positioning  system  and  for  analytical 
continuation  in  space  in  the  case  of  flat  or  moderate  terrain. 

*Xn\provsd  point  positioning  approaching  classical  surveying  accuracy. 

*Flood  plain  profiling  and  mapping  under  consideration  of  the  underlying 
geoldal  structure. 

^Geophysical  prospecting. 

Section  2 of  this  paper  describes  guasi-optimal  and  suboptimal  methods  for  the 
determination  of  C end  n by  means  of  RGS5  and  auxiliary  data.  Section  3 gives 
a short  overview  as  to  required  auto-correlation  functions.  Section  4 outlines 
the  construction  of  regular  n~grid  networks  from  solitary  course  data. 

Finally,  section  S addresses  essentially  in\proved  stochastic  position  error 
control  which  would  be  particularly  valuable  in  the  context  of  local  surveying 
within  a radius  of  about  20  km  from  the  starting  point. 
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Ttie  error  differential  equations  of  interest,  applicable  to  horieontal  motion, 
are  for  the  RiGSSi 


d * 
dt  * 


- i i 

R 


d y 

dt 


iy 


a X - S„«,  - + gn  + «, 


dt  N Z N 


- y ■ ” S_<>_  + gif^  - gS  + a« 

At.  B Z E N 


dt 


't’z  " ^'•'1  ■ Pn  y + ® 


- 

dt  " 


X 

R 


+ a)2*g  + e 


A - 

dt 


R 


+ 0)2*^,  - 01^*2  + Y 


(1) 


(2) 


(3) 


(4) 


(5) 


(6) 


(7) 


For  eiR\plioity,  the  symbol  6 in  front  of  the  dependent  variables  has  been 
omitted.  The  applicable  coordinate  system  is  evident  from  Figure  1. 


(East) 


Figure  1 

Applicable  Coordinate  System 
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Synboli  uiad  in  th«  f ongoing  loenl  lav*],  aquations  arai 
x aaat  angular  position  orrer 
y north  angular  position  error 
X oMt  valeoity  arror 
y north  valoolty  arror 
■ asinuth  axis  angular  drift  rat*  arror  ^ 

0 north  axis  angular  drift  rata  arror  ^ 

Y aaat  axis  angular  drift  rata  arror ^ 
g nomal  gravity 
4 geographic  latitude 
H naan  earth  radius 

asinuth  platforn  attitude  arror 
platfom  tilt  arror  about  north  axis 
platforn  tilt  arror  about  aaat  axis 
gn  ■■  ~ product  of  g and  daflaction  coiqponant^ 


■m 


dVg 

dt 


product  of  g and  daflaction  ooiqponant^ 

north  aooalaration  of  survey  vahiola 
aaat  aooalaration  of  survay  vahiola 


Sg  oorralatad  east  acoelarometar  error 


ag  oorralatad  north  aooalaromatar  arror 


^ 0"T+  (S-?)fY"Y  + (y  + y)  wharo  tha  bar  symbol 

Indloatas  oenstant  bias  and  tha  tarns  in  paranthasaa  are  coraralatad  random 
errors. 

^ T la  tha  earth's  anoawlous  gravity  potential.  The  darivativas  1-  and  ~ 

am  takan  along  tha  level  horison  in  the  eastern  and  northern  direction. 
raspaotively. 
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- ncoa^  north  earth  rate 

p - V /R  north  angular  rate-’^ 

N E 

(li  ~ il  + p ■ ncoa^  + V-/R  north  apatlal  rate 
Wj,  - pg  - -V^/R  eaet  apatial  rate 

vertical  apatlal  rate 


In  inertial  land  navigation,  terms  involving  in  equatlona  (5)  and  (6) 
are  neglected  in  Litton 'a  Kalman  error  controller.  The  initial  oonditiona 

at  tiraa  t,  ■ 0 are  4^°^  ■ 0,  y^®“  y®“  0 . 

The  ayatara  (5)- (7)  can  be  approximately  aolvad  in  cloaed  form  tor  constant  gyroblaaes 

cT,  T,  7*  The  solution  is,  with  u»„  ■ fi  and  i»i„  ■ 

H N Z Z 

♦j(t)  ■ 7co8*  • 0"^  (1  - ooafJt)  + ffiain^  oos#t  - slnOt) 

+dTsln^*  • t + ooa^#  • O'^ainfit)  (6) 
gjj<t)  - Ysin*  • (cosfit  - 1)  + F(cob^*  * t + aln^#  • ti'^sinOt) 

+ aalnf*  0084(t  - n~^Blnnt)  (9) 

^„(t)  ■ yn'^sinnt  + Jain*  n"^  (1  - ooant) 

E 

+ cToob^  n"^  (cosRt  - 1)  (10) 

The  substitution  of  respectively  in  equations  (3) 

• » •• 

and  (4)  permits  the  direct  aaseasment  of  gyro  bias  effects  on  y and  x. 

In  order  to  provide  for  quasi- continuous  time  Integration  for  intervals  X 
between  stops,  aucceasive  representative  C'a  and  n'a  for  constant  time 

^ In  the  error  equations,  normal  and  laerldlonal  radii  with  reapect  to  the 
reference  ellipsoid  may  be  replaced  by  R for  |v|  < 100  km  h~^. 
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Intervals  are  designated  by  a subscript  v.  Subsequently,  niunerlcal  inte- 
gration with  respect  to  tine  yields  the  following  solution 
structure  for  the  first  interval  between  stops,  identified  by  X ■ li 


HA  i +518  n +Ca  + D$  +EY  + Ea  (o  -o)+  Eb  (8  -8)+  c (y  -Y  ) 
Iv  V Iv  V 1 1 1 Iv  V Iv  V Iv  V V 


+ EA,  a„  + EB,  a„  ■ x,  + ,n  ) 

Iv  Nv  Iv  Bv  1 10'  0'  e'  6 


(11) 


Eg  £ + Eh  n + J a + K P + L Y + Ed  (o  -«)+  Ee  (8  -?)+  f (y  -7  ) 
Iv  V Iv  V 1 1 1 Iv  V Iv  V Iv  V 


+ EG,  a„  + EH,  a„  - Y,  + 'C,  (5^,  n„,  ? , n ) 
iv  NV  Iv  Ev  ■'1  10  0 e e 


(12) 


The  terms  involving  a^,  8^>  And  stochastic  accelerometer-induced  errors 
are  only  used  for  the  computation  of  error  covariance  matrices  which  are 
necessary  for  the  establishment  of  a priori  weights  in  a least-squares 
solution.  The  terms  and  represent  knom  linear  functions.  The  small 
terms  ^E^Z  equations  (3)  and  (4),  respectively  are  omitted  prior  to 

the  determination  of  constant  gyro  biases  and  may  be  considered  in  an 
iterative  scheme. 

The  variables  5 and  n except  £ » £ » n » h as  given  initial  and  terminal 
V V 0 e 0 e 

values  are  estimated  by  means  of  statistical  collocation  by  suitably  spaced 

A A 

£^  and  the  number  of  which  should  be  chosen  to  achieve  sufficient  degrees 
of  freedom  in  a post-mission  adjustment.  Accordingly,  under  consideration 
of  representative  £^  ■ £^[x(t),  y(t)],  ••  n^Cx(t),  y(t)], 
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(13) 


5 - 2m  , 5, 

V vi  i 


under  ooneideration  of  5 ■ C . n ■ n . 

0 O'  0 0'  a e'  e e 

For  short  distances  no  advantage  is  gained  from  estimating  and 
In  equations  (13)  and  (14)  by  additional  terms  containing  f)^  and 
respectively. 

A A 

Substitution  of  and  estimates  (13)  and  (14)  In  \ equations  of 
the  form  (11)  and  (12)  yields  the  final  observational  equations! 


'x  ■ “Xi'l  " * “x^  * "x^  * *x’ 


^ AA  AA  AA  A^ 

- (x.  + a,c  + b.n  + c.C  + d.n  ) ■ 0 
A X 0 X 0 X • X B 


7 

^ - ^^1«1  ^ men’ll  + »x“  + 


- (y  + e C + f n + g.e  + h.n  ) - 0 
A AO  Ao  AO  AO 


A weighted  least-squares  solution  yields  quasl-optlmal  deflections  of  the 
vertical  together  with  gyro  biases  and  also  makes  It  possible  to  provide 
error  estimates  thereof.  Approximate  constant  survey  vehicle  velocities 
between  stops  and  standard  vehicle  acceleration  and  deceleration  would 
simplify  the  analyses  and  contribute  to  greater  accurccy. 

The  geometrical  considerations  relating  to  equations  11)  - (14)  are 
evident  from  Figure  2 . 


Figure  2 

Traverse  with  Stops  and  Point  Estimates  and  Segment 


For  the  computation  of  and  n^^  It  is  neoessary  to  record  t*  x,  y,  to 
tabulate  these  data  together  with  stop  and  point  estimation  coordinates 
X(  y,  to  con\pute  distance  and  correlation  matrices 
and  finally  to  determine  m^^  and  n^^-regression  coefficients  under  restriction 

^ a* 

to  five  appropriate  or  n^^oB^^^'^ors.  In  a strict  solution,  corresponding 
m^^'s  and  differ  from  each  other.  Zn  practice,  it  may  be  possible  to 

use  isotropic  correlation  functions  for  both  C end  n because  of  short 
distances  involved  (see  section  3).  Due  to  the  small  correlation  between 

A A 

( and  n for  short  distances  no  advantage  is  gained  by  estimating  end 

A e 

in  equations  (13)  and  (14)  by  additional  terms  containing  and 
respectively. 
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The  present  quaal-optlmal  method  Is  Illustrated  by  the  following:  The 
observable  acceleration  error  at  the  first  vehicle  atop  is 

yi  - - g - 5^)  + (17) 

under  consideration  of  the  initial  deflection  coR\ponent  Zq.  The  tilt  error 
is  then  estimated  by  means  of  the  observable  velocity  errors  and  yj^ 
according  to  the  linear  regression  equation 

^Bl  " “A  + hh 

The  accelerometer  is  then  biased  according  to 

• • 4*  A 

- yi  - g'(>Ei  (19) 

The  deflection  difference  of  interest  is  then  estimated  as 


With  respect  to  the  second  stop  Interval  it  is 

y2  - g (^Ei  - $E1>  + " 9 (Ca  “ e^)  + ^2  ^21) 

where  (S(|)e2  1b  estimated  in  the  form 

^♦e2  “ “2*2  ^2^2 


Subsequently , 

yBa  - h - 9«^2 

The  second  deflection  change  estimate  is  then 
\ DN 

52  - «o  - - 1 r 


(23) 


The  accuracy  of  deflection  determination  is  thus  dependent  on  the  accelerometer 
error,  gyro  bias  error,  and  the  cumulative  tilt  estimation  error. 


with  as  initial  estimates,  it  is 

AO  A 


r , .i  c(i)_  e 


Jl)_  Jl)_  JlK,n^,n-l 


5V  5 +A*,?'*'-  5,+A‘-A*, ,r  C r V+A  -A"  * 

1 0 2 1 n e n-1 

Following  estimation  of  and  n^i  the  closing  errors  ( and  n.-ni  ai^e 

e e e e ^ ^ 

available  for  the  approximate  determination  of  average  gyro  biases  a,  7,  7* 
However,  at  least  one  additional  intermediate  C or  n is  required  for  a unique 
bias  oaloulatlon.  Finally,  bias  oorreotions  to  the  initial  deflection  components 
can  be  applied  as 


In  contrast  to  the  above  procedure,  the  prior  quasl-optlmal  method 

for  the  determination  of  i.  and  n -data  contains  sufficient  degrees  of  freedom 

1 X 

for  the  simultaneous  estimation  of  gyro  biases.  It  is,  however,  of  significance 
that  and  y^  of  the  general  optimisation  method  and  Xj^;^  and  y^;^  of  the  present 
RGSS  mechanisation  are  different  because  of  implemented  Kalman  "corrections" 

It  is,  a. g.. 


• • i^X-1  " 


with  X>2.  While  the  identified  accelerometer  "corrections"  permit  an 
improvement  in  stochastic  position  determination,  they  are  not  beneficial  with 
respect  to  optimal  C,  n -determinations. 

The  present  suboptimization  method  is  due  to  Huddle  11977  ] while  the  quasi- 
optimization concept  was  originally  formulated  by  Baussus  von  Luetzow  [1977]. 
3.  Necessary  Covariance  Functions 


Useful  and  consistent  covariance  functions,  including  cross-covariances,  per- 
taining to  Ag,  i and  n are  those  developed  by  Jordan  [1972],  The  covariance 
functions  aret 

«gg(r)  - Og  ^gg  - o|  d + J ^ 
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(r,  e)  - - Og  (1  + ^ ^ 008^  e) 


ree 


»nn  “ "n^n  ® 

Suitable  constant  parameters  are 


-r/D 


(26) 

(29) 


Og  « o^g  ■ 35  mgal,  ■ 5 arosect  D-37  km. 

Cross-oovarlance  functions > although  available  and  includable  in  the  general 
optimisation  method  outlined  in  section  1,  are  not  shown  here.  For  short 
distanoes*  oroas-oorrelations  tend  to  be  small,  and  they  become  hardly  sig- 
nifloant  for  longer  distances.  The  assumption  of  (approximate)  homogeneity 
applies  to  all  covariauioa  functions.  The  geometry  relating  to  an  arbitrary 
2 point-correlation  Is  evident  from  Figure  3. 

II 


Figure  3 

2 Point-Correlation  Geometry 

Basic,  direction- independent  correlation  functions  (|i  , il*,.  (r,  9)  and 

gg  55 

A , (r,  9)  are  shovm  in  Figure  4. 

^nn 
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Flgura  4 

+ Ag-Correlation  Function  * 

gg 

6— n -Correlation  Fuitction  for  e-0 


0 5 -Correlation  Function 


for  6>0 


covariance  functions  pertaining  to  correlated  gyro  drift  errors  and  correlated 

t 

accelerometer  errors  are  approximated  by  A • e ~ 'f  • Parameters  to  be  used  are 

Oq  " 0.002‘’/hr  and  - 2 hr  in  connection  with  Litton* s G200  gyroscopes,  and 

10  Mgal  and  t « 40  min  in  the  case  of  Litton* s A200  accelerometers. 

A 
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A A 

4 . Estimation  of  Grldded  g,  n-Data  from  Discrete  RGSS- Determined  €,  n- 
Irformation 

For  a mathematioally  satisfactory  solution  of  the  estimation  problem  in 
question,  error  covariances  e 7t^)  a (t^^)  <•  e^^^  should  be  ooir\puted  from 
time-dependent  linear  aggreqates  of  correlated  gyro  and  accelerometer  errors. 

A A 

In  general,  it  is  sufficient  to  estimate  C,  n-data  from  about  20  oorreaponding 
C,  ri-valuaa,  obtained  by  moans  of  the  HGSS.  otherwise,  the  computational 
load  as  to  matrix  inversion  becomes  too  great.  Under  utilization  of  Figure  S, 

A 

(-estimation  at  may  be  formulated  as 


*ll"u**ll’* 
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with  regression  ooeffioients  a^^,  etc.  The  first  line  of  the  resulting 
covariance  matrix  is  then 


(30) 
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of  slgnifloanoe  Is  that  "noise”  covariances  may  be  neglected  as  to  data 
from  different  runs.  The  computation  of  respective  covariances  by  meems 
of  equations  (23)  and  (24)  and  the  inversion  of  the  covariance  matrix 
presents  no  computational  dlfflcvtlties.  Sin\plifioations  as  to  the  in- 
clusion of  variable  noise  covariances  may  be  potentially  possible. 

It  should  be  noted  that  the  midpoint  Ag-estimation  from  Ag-data  measured 
at  two  points  separated  by  a distance  of  3 km  has  a meam  error  of  0.32  mgal 
which  corresponds  to  0.05  arosec.  Although  this  theoretical  estimate  appears 
to  be  optimistic,  the  indications  are  that  gravity  anomalies  and  deflection 
ooniponentB  can  be  well-lntexpolated  in  non-mountalnous  terrain  for  grid 
intervals  Ax  ■ Ay  4 5 km. 

5.  Improved  Stochastic  Position  Error  Control 

The  determination  of  gyro  biases  along  with  that  of  deflections  of  the  vertical 
under  availability  of  initial  and  terminal  deflection  oonponents  Hq, 

makes  it  aloo  possible  to  conipute  position  corrections  5Xg(t^),  6yg(t^).  Term- 
inal position  closure  errors  6x_,  6y„  may  thereafter  be  attributed  to  acoeloro- 
meter  scale  factors,  and  these  closure  errors  are  then  linearly  allocated  as 
additional  position  corrections.  A linear  allocation  appears,  moreover,  to 
be  beneficial  if  gyro  biases  corrections  are  not  explicitly  available. 


The  geomstrlcal  framework  for  terrestrial  inertial  navigation  is  the  applicable 
reference  ellipsoid.  If  the  initial  deflection  ooiqponcnts  ere  known, 

leveling  of  the  platform  at  the  start  of  the  survey  mli  <iion  results  in  the 
terms  g(ri-nQ)  and  -g(C-CQ)  instead  of  gn  and  -gS  in  eg- ations  (3)  and  (4). 

For  the  computation  of  var  x(t^),  var  x(t^),  cov  (x,x)(t^),  etc.,  C and  n 


are  considered  as  rattdom  variables  and  Cq  snd  Hq  as  unknown  biases  with 
var  5 emd  var  n#  respectively.  As  an  illtastration , the  variance  contribution 
due  to  in  the  coniputatlon  of  var  y(t^)  is  then  in  sinvpllfled  form 


:tt) 


vary^  ' (t;^)  - gZ  (/  5dt)z+  gz^^r  5(t^-t^_j^)' 


tl-l 


(32) 


Actually I K and  are  correlated  random  variables ^ and  the  correct  confutation 
would  read 

•0)  •<«  , ‘x  _ 

vuj  (t^).v.ty  (tj)  - f {^{dt  ,33, 

’-X-l 

The  Infllcatlons  of  eg.  (33)  and  of  corresponding  correct  vary^^^  and 
(2 ) * 

cov  (/,y) -terms  are  the  following! 

*For  short  distances,  l.e.,  for  local  surveys  within  a radius  of  30  km 

from  the  initial  departure  point,  the  Inoremantal  position  variances 

and  the  total  position  variance  are  significantly  reduced. 

*Por  short  dlsteunces,  Kalman  filter-confuted  regression  coefficients 

for  position  determination  are  not  optimally  confuted. 

*It  Is  beneficial  for  vertical  positioning  to  facilitate  Initial 

Ag-dg^  elimination  by  appropriate  calibration  under  consideration  of 

a measured  g„. 

0 

The  above  conclusions  are  consistent  with  encouraging  good  AGSS  positioning 
results. 


Identified  correction  terms  involve  the  consideration  of  cov  (trCp)  and 
cov  ( n , tIq ) "Confutations  and  thus  require  to  record  the  coordinates  X(t), 
Y(t)  In  addition  to  X(0),  Y(0).  The  existing  computer  capacity  would 
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have  to  be  moderately  Increased.  Improved  stochastic  position  error  control 
would  be  particularly  effective  In  conjunction  with  the  use  of  improved 
accelerometers,  gyroscopes  and  velocity  quantizers  and  then  approximately 
achieve  classical  survey  accuracies. 

6.  Conclusions 

Quasi-optlmal  determination  of  deflections  of  the  vertical  under  simultaneous 
consideration  of  all  measured  velocity  component  errors  at  vehicle  stops  is 
feasible  and  computer-programmable,  it  has  inherent  advantages  over  the 
present  estimation  technique  and  can  be  generalized  to  a fully  optimal  method 
by  the  inclusion  of  the  whole  set  of  observed  accelerometur  errors  at  stops. 
Solutions  involving  covariances  between  i,  nr  Ag  are  neither  protnising  nor 
economic.  The  construction  of  gridded  n*'<lsta  from  RGSS-determined  estimations 
under  consideration  of  non-statlonary  errors  is  possible  and  does  not  necessarily 
require  longitudinal  and  traverse  surveys.  Actual  position  errors  are  smaller 
than  Kalman-estimated  position  errors,  particularly  for  distances  below  30  km. 

The  computation  of  improved  regression  coefficients  for  position  determination 
is  possible  and  can  be  facilitated  without  expensive  complexity.  Further 
theoretical  research  and  testing  and  the  implementation  of  RGSS  hardware 
improvements  are  eipected  to  achieve  highly  promising  results  in  position  and 
gravity  component  determination  for  various  military  and  non-military  applications. 
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ANALYSIS  OF  VARIANCE  ON  THE  TRADE-OFF  FUNCTION 
RELATING  ACCURACY  TO  SPEED  OF  REACTION 
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ABSTRACT.  The  customary  variable  In  the  analysis  of  variance  la 
a single,  continuous  variable,  presumed  to  possess  the  usual  assumptions. 
In  contrast,  the  variable  of  analysis  here  is  a function, 

Ln[P/<l-P)]-  A + BT,  where 


P is  the  proportion  of  correct  responses,  T Is  the  responsa  time,  and 
A and  B era  parameters  to  be  estimated  from  the  data.  This  response  (or 
trada-off)  function  was  derived  from  observations  on  P vs.  T as  human 
subjects  ware  asked  to  operate  a simple  right  or  left-hand  response  to 
a light  signal  at  different  levels  of  an  altitude  chamber  and  mood  as 
determined  by  a psychological  scale. 


The  analysis  of  variance  of  the  trade-off  function  took  the 
following  form] 


Source 

Mean 

Subjects 

Altitudes 

Hoods 

AxM 

Exp  Error 
Pooled  Devs 


2 

2(s-l) 

2(a-l) 

2(m-l) 

2(a-l)  (m-1) 
2(s-l)  (am-1) 
affls(n-2) 


TOTAL 


amns 


to  ascertain  If  altitude  or  mood  affected  the  trade-off  function. 

I.  INTRODUCTION.  Factors  affecting  aircraft  pilots  have  been 
extensively  chronicled  In  the  literature  on  aircraft  safety.  To  a dismaying 
degree,  these  factors  are  difficult  both  to  define  and  to  measure.  A highly 
bally-hooed  factor  currently  enjoying  a most  lucrative  existence  Is  biorhythm 
whose  effect  on  pilots  Involved  in  aircraft  accidents  has  been  scientifically 
damonstrated  to  be  wholly  fallacious  (Wolcott  1977  a,b).  Our  report  is  concerned 
with  two  factors,  altitude  and  mood,  their  definition  and  measurement,  and 
their  possible  effect  on  reaction  time.  Emphasis  Is  placed  on  a novel  statisti- 
cal analysis,  the  analysis  of  variance  of  a function  (Foster  1962). 


The  opinions  or  assertions  contained  herein  are 
the  private  views  of  the  authors  and  are  not  to 
be  construed  as  official  or  as  reflecting  the 
vlewe  of  the  Department  of  the  Army,  Department 
of  the  Air  Force,  or  the  Department  of  Defence. 
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The  laboratory  experiment  consisted  of  measuring  the  reaction 
time  of  a human  subject  responding  to  two  lights.  If  the  right  light 
would  flash,  the  subject  should  depress  the  right-hand  button  within  u 
specified  length  of  tine  for  a correct  response,  and  similarly  for  the 
left.  An  Incorrect  reaponse  would  consist  of  depressing  the  wrong 
button  or  exceeding  the  time  limit.  Time  limits  were  275,  240,  215, 

180  mill!  seconds.  The  tine  limits  were  balanced  and  randomised  In 
their  order  of  presentation  to  preclude  an  order  effect.  At  each  time 
limit,  the  subject  was  challenged  with  100  flashes.  Simulated  heights 
In  the  altitude  chamber  went  from  0 to  12,000  feet  In  2,000  foot  Intervals. 
While  measured  on  a continuous  psychological  scale  mood  was  condensed  for 
analysis  hare  to  the  three  levels  of  high,  middle  and  low.  The  seven 
altitudes  were  condensed  to  low,  middle,  and  high  for  this  analysis,  thus 
creating  a threa-by-three  factorial  design  for  treatments.  There  were 
eight  volunteers,  each  having  at  least  one  trial  In  each  of  the  three 
by  three  factorial  to  provide  a cmcpletely  balanced  design. 

The  variable  of  analysis  is  defined  to  be  the  function  relating 
the  proportion  of  correct  responses  to  reaction  time.  Because  accuracy 
generally  decreases  as  response  time  Is  decreased— a trade-off  between 
accuracy  and  time,  the  function  relating  these  two  Is  customarily  called 
the  trade-off  function. 

II.  DERIVATION  OF  THE  TRAPE-OFF  FUNCTION.  A variety  of  models 
have  been  used  In  similar  experimentation  by  other  researchers.  A good 
review  of  these  Is  given  by  Wood  (1976)  and  Lappln  (1977).  None  of  these 
was  found  to  be  outstandingly  successful.  The  procedure  followed  hers 
wao  to  elicit  a model  from  consideration  of  the  basic  data.  It  seemed 
to  us  after  trying  many  approaches  that  a plot  of  successive  dlffersncss 
in  the  percent  correct  when  divided  by  change  in  time  and  by  percent 
correct  answers  and  then  plotted  against  performance— this  plot  was  linear 
for  most  trials  and  whan  not.  It  tended  to  be  concave  to  the  right,  as 
shown  In  the  following  diagram: 


When  this  dependent  variable  was  plotted  versus  log  performance, 
the  plots  were  occasionally  linear  but  usually  concave  to  the  left  as 
shown  below. 
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Th*  physical  naanlng  of  thaaa  plots  was  Interprstsd  to  bo  as 
followsi  Improvsaent  rats  rslatad  to  perforoancs  fell  off  at  a constant 
rate  as  perfomance  increased*-  Thus» 

AP/dT  ■ A-  IP 

P 

The  derivation  of  the  trade-off  function  was  aceonplished 
by  approxdnatlng  the  differences  by  dlfferenttalSf  separating  variables 
to  give  **  integrating  by  partial  fractions  to  give  tha 

folloulng  aedeli 


1 kexp(-AT} 

which  la  recognisable  as  a 3-paraaMiter  logistic  function.  Plots  on 
logistic  paper, i.e.  a 2-paraaeter  logistic  by  placing  A/B  <■  1,  were 
generally  linear.  Sous  typical  exuples  of  these  plots  are  given  in 
Figure  1.  Therefore,  the  model  Iia[p/(l>^)]  ■ a -f  bT  was  adopted. 

III.  FUNCTION  AMALTSIS  OF  VAMAMCE  (FAV).  IThat  is  PAVT 
Perhaps  It  is  best  axnplifled  in  an  analogy  to  the  typical  analysis 
varlanea.  Hots  that  the  net  sun  of  squares  for  an  affect  can  be  computed 
b,  .ubtr.ctlB,  . .ort..tl«  t«..  ..tU)»/rt 

to  center  the  data  at  the  mean.  In  the  corresponding  FAV  on  linear  re- 
gression, tha  moments  are  taken  about  the  origin  so  that  not  one  degree  of 
freedom  but  two  are  allocated.  Bowever,  the  average  ragrasslen  through  all 
the  points  is  used  as  the  correction  term  when  distinguishing  between  two 
or  more  linear  regressions  representing  groups  or  treatments,  as  illustrated 
below: 


FAV. 

MEAN  REGRESSION  2 
A vs  B 2 

DEVIATIONS  2(n-2) 
2n 


The  complete  analysis  of  variance  of  the  trade-off  function  is 
shown  in  Tsble  1 in  which  the  degrees  of  freedom  are  allocated  as  indicated 
above;  tests  of  the  effects  of  altitude,  mood,  and  the  AXM  interaction  are 
seen  to  be  not  statistically  different.  However,  the  graphs  of  these 
effects  (in  Figure  2)  show  a distinct  ordering  of  the  effects  as  expected 
if  the  effects  are  real. 
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LOW  altitude  Is  taken  as  the  reference  or  ground  level  compared  to 
which  the  curves  for  the  higher  altitudes  are  displaced  downward  re- 
flecting a depressed  performance.  Similarly  with  respect  to  a good 
mood  (as  reference)  poorer  moods  are  associated  with  curves  with  down- 
ward displacement.  That  both  mood  and  altitude  should  have  the  ordering 
as  discussed  above  under  the  null  hypothesis  has  a significance  probability 
of  1/36  as  shown  at  the  bottom  of  Table  1. 

IV.  COMPARISON  OF  FAV  TO  AN  ALTERNATE  ANALYSIS  ON  EFVICIENCY. 
Efficiency  was  defined  ae  the  proportion  correct  divided  by  the  average 
reaction  time.  This  definition  la  able  to  be  compared  to  the  trade-off 
function  by  considering  the  following  plot  in  which  proportion  of  correct 
answers  is  plotted  against  time. 


efficiency  can  bo  seen  to  be  the  slope  of  this  plot.  Recall  that  the 
logistic  model  plotted  the  logarithm  [percent  correct/percent  wrong] 
versus  time  but  the  line  was  not  forced  through  the  origin. 

For  efficiency,  the  alternate  analysis  consisted  of  a multiple 
regression  In  which  efficiency  as  the  dependent  variable  was  related 
linearly  to  altitude  and  mood  both  In  the  original  scales.  In  that 
anelyslst  both  altitude  and  mood  were  found  to  be  statistically  signifi- 
cant, primarily  because  of  a larger  number  of  trials  and  about  double 
the  number  of  subjects.  However,  the  multiple  correlation  coefficient 
was  found  to  be  less  than  .AS  and  R^  usually  less  than  .20.  Thus,  less 
than  20%  of  the  variation  In  efficiency  was  accountable  by  mood  or 
altitude.  Because  of  the  presumed  advantage  of  the  trade-off  function 
and  its  putative  sophistry.  It  was  something  of  a surprise  to  have  it 
perform  not  as  well  as  the  simple  efficiency  approach.  To  pursue  the 
question  further,  the  Intercept  of  the  trade-off  function  was  plotted 
against  slope  for  each  of  the  72  Individual  regressions  In  the  FAV 
analysis.  Because  these  72  points  were  Incredibly  linear.  It  was  concluded 
that  a very  large  experimental  (not  statistical)  correlation  existed 
between  intercept  and  slope.  It  Is  clear  from  this  vary  high  correlation 
that  virtually  all  of  the  Information  In  the  FAV  analysis  was  contained 
either  In  the  Intercept  or  the  slope  parameter,  but  certainly  not  both. 

Thus  no  advantage  was  derived  from  a two-par amter  model  for  those  data. 
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TABLE  1.  FAV  ON  TRADE-OFF  FUNCTION 


Source 

- df 

SS 

MS 

MEAN 

2 

1700.8420 

SUBJECTS 

14 

492.2826 

35.1630 

ALTITUDES 

4 

8.1274 

2.0318 

MOODS 

4 

8.1038 

2.0259 

A X M 

8 

6.6508 

.8313 

EXP  ERROR 

112 

195.2660 

1.7434 

DEVIATIONS 

432 

398.6270 

.9227 

576 

2803.8996 

ALTERNATIVE  APPROACH  TO  SIGNIFICANCE  PROBABILITY 

1.  MOOD:  Number  of  ways  of  ordering  results  ■ 6 

Number  of  ways  if  effect  exists  ■ 1 
P(Observed  result)  ■ 1/6 

2,  ALTITUDE:  Same  as  above. 

: 1/6  X 1/6  - 1/36 


3.  JOINT  PROBABILITY 
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THE  ANALYSIS  OF  PARTIALLY  FACTORIAL  EXPERIMENTS 


J.  Robert  Burge 

Walter  Reed  Army  Institute  of  Research 


I.  INTRODUCTION. 

It  frequently  happens  In  factorial  experiment*  that  one  or  more  of 
the  factor*  la  of  such  a nature  that  certain  treatment  comblnatlone  are 
identical.  To  llluatrate  this  point,  a problem  commonly  encountered  by 
blood  banks  Involved  in  platelet  transfusion  therapy  has  been  selected 
(Table  1).  The  tvo  variables  of  interest  affect  the  integrity  and 
function  of  platelets.  On*  of  the  factors  Involves  two  different 
methods  of  storing  blood.  Ths  other  consists  of  three  different  times 
of  storage  (including  frash  blood  or  NO  storage),  so  there  will  be  no 
differences  between  storage  methods  at  the  ZERO  level  of  time. 


TABLE  1 

Rslatlonship  Between  Storage  Tims  and 
Storage  Method  on  Platelet  Integrity 

Concentrates  Stored  At 

Stored  For  Room  Temperature  Frosen 


0 hours 

yii 

yi2 

24  hours 

^21 

^22 

48  hours 

y3i 

^32 

This  axperimsnt  falls  into  a class  of  experiments  in  which  the 
treatment  combinations  have  an  appearance  of  consisting  of  a full 
sat  of  factorial  combinations  when  in  fact  this  is  not  so.  Under  such 
circumstances,  when  testing  different  types  of  storage,  the  storage 
method  is  irrelevant  when  a zero  amount  of  storage  time  is  administered. 
Consequently,  there  will  be  additional  degrees  of  freedom  for  error 
arising  from  comparisons  between  identical  combinations  and  correspondingly 
fewer  treatment  degrees  of  freedom.  The  partition  of  ths  treatment 
dsgrees  of  freedom  into  their  separate  components  will  also  be  different. 

Computing  ths  Analysis  of  Variance  Table  for  Experiments  Involving 
Quslitative  Factors  and  Zero  Amounts  of  Quantitative  Factors: 


A short  summary  of  this  type  of  analysis  is  given  by  Addelman  (1974) . 

Let  factor  A (holding  time)  represent  the  quantitative  factor  and 
factor  B (storage  method)  denote  the  qualitative  factor.  If  we  assume 
that  the  first  level  of  factor  A is  the  zero  amount,  the  appropriate 
AOV  table  is  given  below  (Table  2)  where  factor  A has  a levels  and 
factor  B has  b levels: 


Source  of 
Variation 


Factor  A 


Factor  B 
(Method) 


TABLE  2 

AOV  for  an  axb  Partially  Factorial  Experiment 
(one  observation  per  cell) 


AXB 


Controls 
(Exp.  Error) 


1-1  J-1 


Note  that  the  SS  due  to  factor  B is  computed  only  for  the  nonseero 
amounts  of  factor  A.  Similarly,  the  SS  due  to  Interaction  of  the 
two  factors  la  utilising  only  the  noneero  amount*  of  the  quantitative 
factor  A. 


It  may  be  helpful  to  Illustrate  the  Addelman  algorithm  with  an 
example  (artificial  data): 

EXAMPLE  ONE 
TABLE  3 

Storage  Method 


82 

Row  Totals 

^0 

49.5 

61 

110.5 

Holding 

3 

2 

Time 

■^24 

18 

41 

59. 

E 

E 

1-1 

J-1 

Tab 

29 

23.5 

52.5 

Column  Totals 

47* 

64.5* 

111.5* 

Source 

df 

SS 

Y.,  - 222. 


2 

1 


Ti  X S,  1 
A B 


(110. S2  + 59^  + 52.5^)/2  - (222)^ 

~6 

2 2 , 

(47  + 64.5  )/2  - (111.5)^ 

18^  + 41^  + 29^  + 23.5^  - 59^  + 52. 5^ 


» 1,009.75 
■ 76.5625 


47^  + 64.5^  + 115.5^ 
2 4 


203.0625 


Error  1 61^  + 49.5^  - (110.5)^ 

— g— 


- 66.125 


Total  5 49.5^  + 61^  + 18^  + 41^  + 29^  + 23.5^  - (222)2  . 1,355.1 

6 


* (An  aeteriek  wae  utilized  to  Indicate  that  the  totals  Involve  only  the 
nonzero  amounta  of  the  quantitative  factor  - holding  time.) 
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II.  REGRESSION  APPROACH  (An  Alternative  Method). 


The  atanderd  analyale  of  variance  model  for  the  two-way 
claselflcatlon  considered  in  Section  1 is: 

'Pij]  - + + 

Where  + Oj  + 013  ■ 0 

31  + 02  " ° 


Thus  if  p,  o^,  a^t  3^i  and  621  knom  or  estimated,  all  other 

parameters  or  their  estimates  can  be  found  from  the  restrictions. 
We  can  write  the  regression  model: 

'D'l.il  ■ “ * “A  * °2*2  * ®A  * ‘2A  ® 

Where: 


fv  1 
*11 

1 

1 

0 

0 

0 

B M 

y 

^12 

1 

1 

0 

0 

0 

and  S •• 

Me 

“1 

Si 

; X - 

' -s# 

1 

.0 

1 

1 

1 

”2 

Y22 

1 

0 

1 

-1 

-1 

s 

Si 

1 

-1 

-1 

1 

-1 

Si 

.S2. 

1 

• 

-1 

-1 

-1 

1 

> 

^21  “*■*22"° 


Si  S2 
Si  + Si 


0 

0 


S2  + S2  “ 0 


no 


^ I 'ifl-  iiMi'iiiiysifiMtfriiaaMfcYisasFiTiiiiigHhVi'rTi' 


One  should  note  the  following: 


(1)  For  all  observations  belonging  to  the  zero  level 


(11)  The  elements  of  the  69]^  column  are  obtained  as  a product 
of  the  corresponding  elements  of  the  02  and  columns. 


(Ill)  Because  of  the  orthogonality  In  X,  we  can  obtain  separate 
orthogonal  sums  of  squares  which  are  additive  for  the  estimates  of 
p,  and  02  > and  622^*  These  wll.l!  be  the  usual  sums  of  squares 
for  the  mean,  rows,  columns,  and  Interaction. 


(Iv)  Two  column  vectors,  and  S,  are  0 
their  Inner  product  Is  zero  (l.e.,  a’^  ■ 0). 


orthogonal  if  and  only  if 


Using  the  R(  ) - notation  for  reductions  In  sum  of  squares  when 
fitting  linear  models: 


A more  complete  summary  of  R(  ) - notation  Is  given  In  Searle(1971) . 


The  R(  ) - notation  Is  defined  by  denoting  as  R(b)  the  reduction 
In  sum  of  squares  due  to  fitting  the  familiar  linear  model 


■ 2 b 


Let  b be  any  solution  to  the  normal  equations 
X'Xb®  - X*Y 


say  b®  - (X'X)**  X*Y 


where  (X'X)"  is  a generalized  Inverse  of  X'X,  meaning  that  It 
Is  a matrix  satisfying  X'X(X'X)"  X'X  ■ X'X,  then 


b®  X’Y  . 


It  can  also  be  expressed  as 


R(b)  - Y'XCX'X)"  X'Y 


we  take  (4)  and  (5)  as  our  formal  definition  of  R(b). 


Now  suppose  b is  partitioned  Into  two  vectors  b^  and  b2 
so  that  the  model  is  (full  model) 


'fXl-  Sifei  + Xjij 


i 4 ! 


I • ^ 


The  reduction  In  sum  of  squares  for 
R(b^,  b^)  - Y'(X^  X2)  1'3^'Xi 

This  being  the  direct  analogue  of: 
R(b)  - Y'X(X'X)“X'Y 


fitting  this  is  denoted  by: 


In  connection  with  (6)  consider  the  submodel  (reduced  model) : 

®rxi  ■ *1  ii  (B) 

For  the  fitting  of  this  model: 

R(bj^)  - Y’X^CX^^Xj^)-  x'y  (9) 

Differences  between  reductions  in  sums  of  squares  are  .also  accommodated 
by  the  notation: 

R(b2  |b^)  - R(b^,  b^)  - R(bj^)  (10) 

This  indicates  the  reduction  in  suns  of  squares  due  to  fitting  the 
full  model  over  and  above  that  due  to  fitting  the  reduced  model. 

The  R(  ) - notation  is  quite  general  and  can  be  used  for  regression 
models,  for  familiar  linear  models  involving  main  effects  and  interactions, 
and  for  combinations  of  the  two. 


In  order  to  demonstrate  this  notation,  it  has  been  applied  to  the  data 
given  in  Table  3 and  summarized  in  Table  4. 

TABLE  4 

AOV  for  a 3x2  Partially  Factorial  Experiment 
(One  Observatlon/Cell) 


Source  of 
Variation 

df 

SS 

R(  ) - Notation 

y 

1 

8,214.00 

R(li) 

Factor  A 
(Time) 

2 

1,009.75 

R ("i.  -zlu.  6^,  Sjj)  . 

» 

' R(a^i  02^ 

Factor  B 
(Method) 

1 

76.56 

R (bJ  y.  a^,  02.  «21^ 

- R(Bi) 

A X B 

1 

203.06 

R (621]  y,  02*  ®iJ 

■ "‘ri’ 

Full  Model 

5 

9,503.375 

R p ^2*  ^1*  ^21^ 

• 

Error 

1 

66.125 

Y*Y  - R (y, 

*^21^ 

Total 

6 

9,569.5 

Y'Y 
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PARTIAL  FACTORIAL  EXPERIMENTS  WITH  MORE  THAN  ONE  ObSERVATlON 


PER  CELL. 


A.  BALANCED  CASE 


Whenever  the  eacperinentel  plen  involvee  one  obeerveclon  in  each 
Bubcleaa*  Addelnenn's  article  applies.  The  following •exaaple  Illustrates 
how  to  treat  a 3x2  partial  factorial  exparlnent  when  cell  frequencies 
exceed  one.  Specifically,  the  sero  treatnent  has  been  replicated 
d times.  All  of  the  other  subclass  frequencies  equal  3. 


EXAMPLE  TWO 

(Cell  Frequencies  Greater  Than  One) 


Storage 

1 Storage  Method 

Row 

Tima 

®1 

^2 

Totals 

TaS  0 hours 

63 

49.5 

220.5  - 

Y, 

u 

61 

47 

Xe  ♦ 

T. : 24  hours 

23 

40 

1 

13 

41 

18 

42 

54 

-^11. 

123  • 

"12. 

177  » 

"2 

dm  • s 

Ta : 48  hours 

40 

22 

2 

18 

23.5 

29 

25 

87 

-^21. 

70.5 

• 

CM 

1 

157.5  - 

Column 

Totals 

141 

-^*1. 

193.5 

-"*2. 

<14/  e ..  V 

555  - T 


Analysis 

of  Variance 

Computations: 

Factor  A 

2 

9 9 

8S(T) 

- 220.5 

+ 177  + 157.5^  - 

(555)*'  - 2,259.375 

4 

6 6 

16 

Factor  B 

9 9 

S8(S) 

- 141^  + 

193.5*'  - (334.5)^ 

- 229.6875 

6 

12 

i I 

’ I 

i m 


i J 


I 


Interaction 
SS 


■ 54*  + 87*  + 123*  +70.5^  - 177*  + 157.5*  - 141*  + 193.5 


+ 

f334.5)  • 609.1875 

12 

Error 

63*  + 61*  + 49:5*  + 47*  - (220.5)*  +•••+  22*  + 23.5^  + 25 

4 

- 

(70.5)*  - 492.6875 

3 

Mean 

m 

(555)*  - 19,251.5625 

Total 

- 

63*+"*+  25*  - 22,842.50 

TABLE  5 

AOV  for  an  3x2  Partially  Factorial  Experlnant 
(Call  Frequenciea  Exceed  One) 

Source  of 
Variation 

df  88  R(  ) - Notation 

Factor  A 
(Tlaa) 

Factor  B 
(Method) 

AxB 


Full  Model 
Error 


1 

19,251.562 

R(y) 

2 

2,259.375 

R(Oj,  0 

1 

229.6675 

R(8j|m, 

1 

609.1875 

R(6,,iu 

®21^  ■ ®2|V7’‘  “2' 


«2l)  - R(Pi) 


P,)  - R(«,J 


22,842,50 


B.  UNBALANCED  CASE 


The  analysis  of  variance  (AOV)  table  has  been  used  to  summarize 
the  results  from  the  snslysss  performed  up  to  this  point.  The  results 
from  an  enelysie  of  balanced  date  ere  frequently  summarized  in  this  way 
because  researchers  automatically  associate  each  sum  of  equaree  (SS) 
line  item  In  the  table  with  testing  a particular  hypothesis  in  the 
linear  model. 

In  the  axb  partially  factorial  experiment  with  one  observation  per 
cell,  the  regreseion  approach  offers  a computationally  convenient 
alternative  for  generating  the  verioue  entries  in  the  AOV  table.  This 
situation  is  analogous  to  balanced  data  (equal  subcleas  frequencies), 
so  there  is  no  confusion  as  to  whet  is  being  tested  in  each  line  item  of 
the  table.  However,  when  the  cell  frequencies  exceed  one,  as  they  did  in 
example  two,  a complication  ensues  (vis.,  R(a.,  02)  oiq  I P)  ••  the 

result  of  an  emerging  non-zero  off-dlegonal  element  in. the  X’x  matrix). 

The  situation  becomes  more  complicated  when  further  unbalance  arises 
through  missing  observations.  With  unbalanced  date  hypotheses  tested 
under  such  headings  as  row,  column,  and  interaction  effects  cannot  be 
uniquely  presented  by  paralleling  (merely  extending)  the  methods  of  analysis 
for  balanced  data. 

In  order  to  illustrate  these  points  considsr  the  following  example: 

EXAMPLE  THREE 


Hissing  Observations 

Storage 

Storage  Method 

Time 

SI 

S2 

Tn  0 hours 

61 

49.5 

T..  24  hours 

23 

41 

13 

(41)  t 

T.,  48  hours 

(40)  t 

22 

18 

25 

t The  two  numbers  in  parentheses  will  be  treated  as  missing  observations. 


W«  can  write  the  regression  model  E 


In  the  situation  with  unbalance  created  through  mlaalng  observatlona, 
the  two  rows  of  the  X matrix  corresponding  to  the  miaaing  observations 
(linaa  drawn  acroas  thda)  will  be  delated. 

The  upper  triangular  X'X  matrix  baaed  on  all  10  observations  1st 


X'X(IO) 


While  In  the  case  of  2 missing  observations  wa  have: 


X'X(8)  - 


8 -1 
5 


In  ganaral,  for  a 3x2  partially  factorial  experiment  with  the  following 
call  frequencies i 


Stored  Storage  Method 

For  St  S. 


Row  Totals 


Column 

Totals 


IE; 


X'X 

will  have 

the  following 

structure: 

V 

“l 

“2 

»1 

*21 

M 

^1  - ^2 

Em 

*.2 

^22  ■ ^12 

^12  " ^22 

f 

• a 

ora 

f 

^l-^2 

f 

a a 

«h«r«  N ■ Iq  + + fj 

and  £ ■ £,  + £« 

••  1*  ** 


Tha  affact  of  unequal  aubclaaa  fraquenclea  pruducea  aun  of  aquarea 
that  ara  not  orthogonal.  Thua,  tha  influanca  of  alaaing  obaarvationa 
can  ba  raadlly  obaarvad  by  axaalnation  of  tha  off-diagonal  taraa  In  tha 
abova  ganerallzad  X'X  aatrlx  for  a raparaaaterlaad  3x2  partially 
factorial  axparlmant. 

Tabla  6 allowa  ona  to  examine  the  varloua  reductlona  In  auma  of 
aquarea  that  ara  eaally  generated  aa  a reault  of  the  R(  ) -notation. 

The  problem  now  becomaa  ona  of  relating  the  aaaoclated  auma  of  aquarea 
to  the  teatlng  of  "appropriate"  hypothesea  about  the  parametera  In  the 
model.  Obvloualy.  care  muat  be  taken  ao  aa  not  to  Incorrectly  deacrlbo 
what  la  being  teated.  To  better  underatand  how  thia  can  be  achieved, 
haa  promptad  a number  of  artlclaa  In  the  recent  atatlstlcal  literature. 
Howaver,  It  la  recommended  that  you  turn  firat  to  Speed,  Hocking,  and 
Hackney  (1978) . 
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AOV  Table  for  3x2  Fartlallj  Factorial  Experlaent 
lUth  Missing  Observations  (Obeqoal  Subclass  Frequencies) 
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STATISTIC/L  ANALYSIS  OF  EXPERIMENTS  IN  SOBPTIVITY 

Richard  N«  Macnalr 
Edward  W>  Ron,  Jr< 

U.  S.  Amy  Natick  Raaaarch  and  Davalopmant  Comoiand 
Natick,  Maoiachuaatta  01760 

Aba tract.  Thla  papar  daacrlbaa  savaral  axperlmanta  about  the  apparatus  and 
procadura  uaad  In  tasting  matarlals  from  which  chamlcal-protactlva  clothing 
Is  mada.  Tha  purposa  of  tha  work  Is  ultlmataly  to  rsduca  tha  variability  and 
cost  of  such  tasting  by  claarar  undarstandlng  of  tha  procassas  Involvad.  Tha 
daslgn  and  analysis  of  thraa  axparlmants  ara  outlined,  and  soma  tentatlvs 
conclusions  ara  stated. 

I.  Introduction.  This  papar  has  to  do  with  tha  military  problem  of  defending 
a soldier  against  attack  by  chemical  agents  (usually  poison  vapor).  This  da- 
fanaa  takas  tha  form  of  special,  protective  clothing  which  absorbs  large 
quantities  of  ths  agent.  For  this  purposa  large  rolls  of  matsrial  from  which 
tha  clothing  Is  made  arc  purchasad  and  samples  from  thass  rolls  ars  tea tad  for 
their  Borptlvlty,  l.e.  thslr  absorbing  power.  This  tasting  is  slow,  expansive 
and  somewhat  unreliable.  Tha  present  paper  is  a study  of  ths  tsst-mathodology 
(apparatus  and  procadura)  with  tha  objective  of  improving  tha  process,  In  tha 
oansa  of  obtaining  greater  reliability  or  lower  cost. 

Specif loally,  a certain  standard  test  procadura  and  apparatus  has  bean 
ussd  In  the  past.  Recently,  a simpler  apparatus  of  the  same  general  type  has 
been  triad  and  found  to  give  acceptable  reaults  in  a ahorter  time  than  the 
.tandard  method.  However,  the  results  wars  Irregular  enough  to  raise  questions 
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about  tha  antira  procaaa,  ao  It  was  thought  daslrabla  to  conduct  tha  mora 
systematic  tests  dascrlbad  here. 

II.  Background.  Tha  two  apparatuses » sketched  schematically  In  Figure  1» 
are  generally  similar.  Instead  of  a toxic  agent  CCL^  is  used  as  a repre- 
sentative vapor  in  these  tests.  A calibrated  mixture  of  this  vapor  and  an 
Inert  gas  flows  through  the  sample  at  a standard  rate,  and  a test  for  de- 
tection of  CCL^  is  done  on  the  downstream  side  of  the  sample.  The  result 
of  each  teat  Is  the  length  of  time  (In  minutes)  before  detectable  CCL^  ap- 
pears, l.e.  before  "breakthrough".  The  sorptlvity  or  amount  absorbed  at 
breakthrough  Is  then  calculated. 

The  two  vapor-penetration  apparatuses,  which  we  describe  as  standard 
(std)  and  simplified  (sim),  reapeotivaly,  differ  In  the  following  respects t 
<a)  Nitrogen  (M2)  is  used  as  the  Inert  gas  In  the  s^d  and  air  in  the 

aim. 

(b)  On  the  upstream  aide  of  the  sample  the  gas  is  heated  to  32®C  in 
std  but  maintained  at  ambient  (usually  20®C)  in  the  aim,  The  tests  are 
both  conducted  in  a room  with  coarse  temperature  control  but  no  humidity 

regulation. 

(c)  The  methods  of  detecting  CCL^  on  the  downstream  side  of  the  sample 
are  notably  different,  but  both  Involve  a human  observar.  In  std  the  gas 

la  pyrolised  and  bubbled  through  a tube  containing  a starch-potassium  Iodide 
solution,  which  turns  blue  when  products  of  CCL^  ars  present.  In  sim  the 
gas  flows  over  a copper  disc  in  the  flame  of  a propane  torch,  and  the  flame 
turns  green  in  the  presence  of  CCL^. 

(d)  The  human  operators  observe  the  color  changes  in  the  two  apparatuses. 
For  std,  the  observer  records  the  time  at  which  the  liquid  in  the  bubbler  tube 
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first  shows  blue.  For  simi  he  records  the  time  at  which  the  flame-color  Is 
perceptibly  "more  green"  than  a comparison  flame  located  near  the  test  flame 
but  not  In  the  flow-stream.  An  additional  complication  Is  the  following i 
It  is  thought  (based  on  previous  experience)  that  the  std  tends  to  give  false- 
positive readings.  The  operators  are,  therefore,  instructed  to  replace  the 
bubbler  tube,  when  the  first  blut  color  is  observed,  with  a fresh  bubbler 
tube.  The  time  Is  recorded  only  If  the  fresh  bubbler  tube  shows  blue  within 
two  minutes.  Otherwise,  the  fresh  bubbler  tube  Is  Isft  In  place  until  It 
shows  blue.  This  process  is  repeated  as  many  times  as  necessary  until  a blue 
color  is  obtained  within  two  minutes.  No  analogous  routine  Is  applied  to  the 
slm. 

III.  Object Ives » We  would  like  to  answer  the  following  questions  about  these 
test  methods; 

(1)  Do  the  apparatuses  give  the  same  results? 

(2)  Is  there  an  effect  due  to  the  Initial  concentration  of  the  agent  CCL^? 

(3)  Do  the  operators  have  an  affect  on  the  results? 

(A)  What  other  factors  am  important? 

Concerning  (1)  and  (2),  If  the  apparatuses  and  concentrations  have  an 
effect,  then  we  want  to  know  whether  some  combination  of  slm  and  con- 
centrations gives  results  that  can  be  reliably  used  in  place  of  the  std  at 
Iti  traditional  concentration.  Concerning  (4),  some  other  factors  that  may 
have  effects  are  material  properties,  ambient  conditions  and  the  treatment 
of  the  material  samples  prior  to  testing. 


IV.  Variables.  The  variables  In  the  teats  are  as  follows: 

A,  apparatus,  takes  on  values  (1)  slm,  and  (2)  std. 

C,  concentrations  takes  on  values  (1) , low  or  5 mg/l,  and 

(2),  high  or  15  mg/l. 


D,  day  of  teat  1 to  4. 

G,  sample  group,  (1)  and  (2). 

0,  operator,  (1)  and  (2). 

R,  repetition  number,  1 to  12. 

W,  week  of  test,  1 to  2 or  3. 

V.  Design  of  Testa.  Three  different  teats  in  all  were  run. 

Test  li  In  this  teat  each  operator  remained  with  the  same  apparatus 
throughout  the  experiment.  To  mlnlmiea  the  effects  of  material  variability, 
samples  were  re“UBed  over  and  over  again.  In  order  to  restore  samples  as 
nearly  as  possible  to  their  original  condition,  they  were  heated  overnight 
In  an  oven  at  50®C  prior  to  each  day's  tests  (including  the  first  day's). 

We  surmised  that  this  cycle  of  heating  and  testing  might  affect  sorptivity, 
so  the  test  plan  Involved  periodic  repetition  of  tests  in  order  to  find  and 

eliminate  any  trend, 

Twenty-four  samples  were  taken  as  close  together  as  poeslble  from  a 
large  roll  of  material  and  divided  into  two  groups  of  twelve  samples  each. 
The  groups  ara  labelled  1 and  2 and  chosen  in  the  irregular,  but  not  random, 
sample-pattern  shown  in  the  sketch. 
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Th«  t««t  plan  for  aach  weak  ia  aa  follow*: 
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Thla  taat  waa  run  for  two  waaka,  than,  after  a lapaa  of  about  a month > 
for  a third  weak. 

Taat  2t  Thla  waa  naa;:ly  a repetition  of  Taat  1.  The  only  differences 

ware: 

(I)  A different  aat  of  24  aamplaa  waa  used  from  the  large  roll. 

(II)  The  teat  laatad  only  two  waaka. 

Taat  3;  Thla  dlfforad  aubatantlally  from  tha  two  preceding  teata.  Samples 
ware  not  ra-u*ad,  and  f.ia  affect  of  operator  waa  apaclflcally  Investigated.  The 
aamples,  96  In  all,  ware  obtained  from  two  aheata  of  the  aama  large  roll  used 
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in  Tests  1 and  2.  Each  sheet  was  divided  Into  6 rectangular  set8»  and  8 
samples  were  taken  from  each  set.  From  each  of  these  12  sets,  one  sample 
at  a time  was  randomly  chosen,  and  these  12  samples  formed  a group.  In 
this  way  8 groups  of  12  samples  each  were  obtained  and  tested  according 
to  a plan  that  was  Identical  with  that  of  Test  1 except  that  the  variable 
0 replaced  the  variable  G. 

Teat  3 differed  from  Tests  1 and  2 In  one  other  Important  respect. 

Since  the  samples  were  not  re-used,  there  was  no  need  to  heat  them  overnight. 
Consequently,  they  were  simply  kept  at  65Z  relative-humidity  for  several 
days  prior  to  test.  This  la  a much  higher  humidity  (about  600Z)  than  that 
of  the  samples  heated  In  the  oven  prior  to  test,  which  was  about  101. 

VI.  Results  and  Discussion.  The  statistical  analyses  of  the  data  were 
done  using  principally  the  routines  of  the  Statistical  Package  for  the  Social 
Sciences  (SPSS),  edition  2.  The  main  results  are  shown  In  Figures  2 and  3 
as  graphs  of  sorptlvlty,  y,  versus  various  variables. 

Figure  2 shows  graphs  of  the  daily  averages  of  the  results  for 
Experiments  1 and  2.  The  main  reason  for  studying  these  is  to  see  whether 
there  is  any  overall  trend  in  the  tests  where  samples  were  re-used.  No 
significant  quadratic  effects  were  found.  Weak  but  significant  (95X  level) , 
and  nearly  equal,  downward  linear  trends  were  found  In  the  data  from  Teat  2 
and  the  first  two  weeks  of  Test  1,  labelled  (1^)  in  Figure  2.  When  the 
third  week's  data  was  appended  to  Test  1,  the  trend  disappeared. 

Since  about  a month  elapsed  between  the  second  and  third  weeks 
of  Test  1,  it  is  not  clear  how  to  inter r*et  the  trend.  The  most  plausible 


explanation  la  that  re-heatlng  has  an  effect,  hut  that  effect  was  washed  out 
by  the  simple  passage  of  tine  between  weeks  2 and  3<  In  any  case  the  trends 
are  rather  weak,  and  nc  further  effort  was  made  to  correct  for  themi 

Figure  3 shows  the  effect  of  apparatus,  A,  and  concentration,  C,  on 
the  sorptlvlty  In  the  form  of  estimated  means  and  95Z  confidence  limits  for 
the  three  different  tests. 

Figure  3(i)  shows  that  the  three  testa  gave  Inherently  different 
results.  Tests  1 and  2 differed  only  slightly,  but  Test  3 gave  lower  and  more 
variable  results.  Possibly  these  differences  are  due  to  both  material  vari- 
ability and  the  different  pre-conditioning  of  the  samples  In  Test  3. 

Figure  3Cli)  and  3Cill)  nhow  the  main  effects  of  A and  C on  sorp- 
tivlty.  Clearly,  the  std  A produces  higher  readings  than  the  aim  A,  the  size 
of  the  difference  being  much  greater  In  Test  3 thnn  In  the  others.  The  ef- 
fect of  C Is  less  clear.  Tests  1 and  2 show  no  difference,  but  Test  3 sug- 
gests that  higher  C gives  lower  sorptlvlty. 

Figure  3(iv)  exhibits  the  interaction  between  the  effects  of  A and 
C.  These  are  somewhat  confusing.  First,  we  notice  that  in  Test  1 there  was 
no  effect  of  A at  low  C,  l.e.  the  overall  effect  of  A was  due  entirely  to  the 
effect  at  high  C.  Related  to  this  is  the  fact  that  in  Teste  1 and  2,  the 
result  of  increasing  C is  to  increase  the  sorptlvlty  measured  by  std  A and 
decrease  that  by  slm  A.  This  effect  Is  not  observed  in  Test  3,  and  is  not 

always  significant  in  Tests  1 and  2. 

An  ANOVA  of  Test  3 showed  that  there  was  no  main  effect  of  operator. 
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Howeveri  operator  1 gave  consistently  higher  readings  on  the  std  A and  operator 
2 on  the  aim  A. 

Most  of  the  statistical  testa  (F-tests,  t~tests)  are  based  on  assumed 
normality  of  the  data.  Some  rough  estimates  of  departure  from  normality 
(skewness  and  kurtosls)  were  made  for  the  three  sets  of  data.  The  data  of 
Test  1 were  not  perceptibly  non-normal;  Teats  2 and  3 were  less  satisfactory 
but  not  outrageously  non-normal. 

It  was  thought  that  there  might  be  some  effect  of  the  time  of  the  day 
on  the  results,  but  a plot  of  the  overall  means  for  each  repetition  number 
showed  no  effect  that  could  not  be  attributed  to  randomness. 

VII.  Conclusions. 

The  results  of  these  teste  are  somewhat  fragmentary  and  suggest  that  our 
understanding  of  the  process  Is  Incomplete.  One  thing  which  emerges  Is  that 
the  effect  of  sample  pre-conditioning  is  very  important.  Generally,  the 
std  A gives  higher  results  than  the  slm  A,  the  size  of  the  difference  being 
much  affected  by  C and  by  the  preconditioning.  The  std  A gives  somewhat  more 
stable  values  than  the  slm  A. 

Tests  In  which  one  tries  to  observe  the  first  non-zero  value  of  a con- 
tinuous variable  are  notoriously  unreliable.  Some  of  the  present  difficulties 
might  be  avoided  by  using  comparisons  with  standard  color  cards,  or  some 

Obviously  much  remains  to  be  done  in  this  area. 


spectral  analysis. 
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ON  CONBININa  I’SEUDO-RANOOM-  NUMHBR  OBNERAIORS 


Murk  Brown and  Harbart  Solomon 


SUMHARy 

A tadhniqua  uaad  in  paaudo-raadoB  numbar  ganaratlon  la  to  oombina  two 
or  Bora  dlffarant  ganara.tora  with  tha  goal  of  produolng  a naw  genarator 
with  laqprovad  randonnaia  propartiea.  Va  atudy  audi  a olaaa  of  ganaratora 
and  ahow  that  In  a atrong  aanaa  tha  oombinad  ganarator  doai  offar 
iaprovamant.  Our  approach  appliaa  raculta  froB  majorlaatlon  thaoxy. 
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On  Coublnlug  Paeudo-Bandou  Nutober  Qonaratora 


Mark  Brown  and  Haibavb  Solomon 


1.  Introduotion. 

Many  methoda  have  been  7rc>;^aed,  teated  and  employed  for  generating 
paeudo-random  nutfbera  ([2],[3]>tlt-],[5]#[8]/[9]>[ll]>[12],[l4],[l6],[l8], 

[19] )i  (Rie  goal  la  to  produce  atrlnge  of  numbera  vftilch  behave  like  Inde- 
pendent uniform  [O^l]  random  variables.  !]ha  generatora  yield  Integers  In 
the  set  (0,l,...,m-l}>  ^Ich  are  then  transformed  to  [0^1]  by  dlvdaion 
by  m.  Suppose  that  and  are  strings  of  numbers 

generated  by  two  separate  generators.  Various  suggestions  have  been  made 
for  combining  the  two  strings  to  produce  a new  string  vAiloh 

hopefully  Improves  upon  X and  Y.  One  method;  discussed  In  Knuth  [S], 
p.  26-27;  !•  to  set  ■ Xj^+Y^(mod  m).  Another;  due  to  Maoloren  and 
Marsoglla  [111;  which  Knuth  reports  to  be  excellent  ([6];  p.  3l);  uses  the 
y string  to  randomly  permute  the  X string. 

For  the  additive  generator  ■ X^+Yj^(mod  m)  we  obtain  the  following 
result  (Corollary  2).  For  any  k and  corresponding  cboice  of  Indices 


1^  < ig  < ...  < ij^  consider  the  veotors  X^ 


X A X A 

denote  the  respective  distributions  of  and  P^;  and  s^^ 

are  probability  distributions  on  "fiX.  ^ere  -/tlm  (0;l;...;m-l}.  Define 

to  be  the  uniform  distribution  over  7>t  » rj^  is  a vector  of  components 

Ir  K 

each  equal  to  m"*.  Let  ll*!!  be  an  arbitrary  symmetric  norm  on  R®  ((|x||  ■ l|Hicl| 
\diere  Ibc  Is  ary  permutation  of  x).  Then  ||s^-rj^|l  < min(l|p^-rj|, 
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For  the  generator  suggested  by  Maclaren  and  Marsaglla  a similar  but 
vesker  result  is  obtained.  Using  Y to  shuffle  results  in 

improvement  for  the  Joint  distribution  of  but  not  necessarily 

for  the  narginel  distributions  of  subsets, 

Ihe  potential  value  of  our  approach  is  that  it  oan  provide  additional 
Justification  for  some  generators  currently  in  use^  axxd  pezhaps  suggest 
new  generators  (Azloh  would  then  be  analyzed  by  traditional  methods. 

In  our  analysis  we  treat  the  strings  X and  Y as  independent  random 
vectors.  In  practice  X and  Y are  deterministic  strings  of  numbers, 
ntils  creates  a problem  in  the  strict  application  of  our  results  to  pseudo- 
random number  generation. 

S.  MaJorlaatlon. 

By  definition  ([6]  p.  4?)  an  n-vector  a is  said  to  be  majorized  by 
an  n-veotor  b if  upon  reordering  to  achieve  a^^  > a^  > • • * > a^  and 

> bp  > ••  • > b^  it  follows  that  E ^ *1  S S 1 It  ■ 

and  E *1  ■ S 1 A function  t/  R”  + R#  defined  to  be  Behur 

convex  <’[131  p«  1189)  If  whenever  a is  majorized  by  b,  i^(a)  < \|f(b). 

Sohur  ooTivex  functions  include  symmetric  convex  functions  Which  in  turn 
include  symmetric  Kauge  functions  end  syrametrlo  norms  ([1],  p.  229).  'Sv 
a symmetric  norm  on  r"  we  mean  a function  IHI,  R*  satisfying: 

||x||  > 0 for  all  X € R*'  with  equality  if  and  only  if  x « 0, 

lIcKxll  « |a|||x||  for  all  asR,  x « l!x+y||  < ||x||  + ||y||  for  all  x,y£R”, 

and  llxll  - ||Rx||  for  all  x e r”  and  for  all  permutations  llx  of  x.  We 
note  that  if  r la  the  uniform  distribution  over  {lj2,.,.,n)  (r(l)  - p 
1 « l,,..,n)  and  I'll  is  a symmetric  norm  on  R**,  then  g(x)  - ||x-r|| 
is  a symmetric  convex  function  and  is  thus  Behur  convex.  Some  references 

for  majorlzatlon  eu'e  [1]/  t8]>  [lj5)>  and  [17]. 
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Lemma  1 below  contains  four  equivalent  statements  relating  to 
majorlzatlon.  Hie  equivalence  between  (l)  and  (11)  Ib  due  to  Hardy, 
Llttlewood  and  Polya  ([6]  p.  the  fact  that  (11)  Implies  (111) 

Is  found  in  [l]  p.  1613  and  (ill)  (11)  In  [1]  p.  I6l)  the  fact 

that  (i)  •«>>  (iv)  la  the  definition  of  Sohur  convexity  and  (Iv)  (1) 

beoauie  'if where  x^^<^  Is  the  1^  largest 
component  of  Is  symmetric  and  convex^  and  therefore  Sohur  convex. 

Lemma  1.  The  following  statements  are  equivalent) 

(1)  a la  majorized  by  b 

(11)  a H Fb  vAiere  F la  doubly  stochastic 

(111)  a Is  a mixture  of  permutations  of  b,  l.e.^  a ■ p^(n^b) 

^esre  & probabllii^  vector  and  each  n^b  Is  a permu- 

tation of  b. 

(Iv)  t(»)  < t(b)  for  all  Schur  convex  funotlons  \|f. 

Theorem  1.  Suppose  that  X Is  a discrete  random  variable  taking  values 
In  the  set  ^ ■ (xj^,...,x^)  with  probability  distribution  p >■  (Pj^/».»>Pjj), 
vAiere  p^  w P(x^)i  and  Y la  a random  variable,  Independent  of  X,  taking 
values  in  the  set  tj,.  For  each  y e let  be  a 1-3.  transformation 
of  ^ onto  Itselfi  Define  Z •»  T.^  and  let  a be  the  distribution  of  Z* 
Then  a Is  majorized  by  p. 

Vroot.  Since  T Is  1-1  and  onto  the  distribution  of  T X Is  a permuta- 

y Y 

tion  of  Pi  Thus  s Is  a mixture  of  permutations  of  p.  By  lemma  1 s 
is  majorized  by  p.  || 


1 


1 


3.  Applications  to  ■pBeudo -random  xmttiber  Beneratlon. 

Suppose  that  X «»  (X^,...,Xjj)  and  Y are  Independent  rwidom  vectors, 

vlth  each  X^  assuming  values  in  '-M  (0,1,..., m-1).  Consider  a s\A>set 

of  k indices  A « (l  < ij_  < Ig  < ••  • < ij^  < Nl.  Define  to  be  the 

distribution  of  ■ (Xj^  )|  Is  a probability  distribution  on 

X Ic 

A k 

lii  . For  etch  y In  tha  aupport  of  Y lat  T be  a 1-1  tronaforoation 

¥ 

of  onto  and  let  s^  denote  the  distribution  of  TyX. 

Define  r^^  to  be  the  uniform  distribution  over  (rj|.(x)  « m"**^  for 
each  X s 7?L^). 

Corollary  1.  Let  s^  and  p^  be  as  defined  above.  !ihen  s^  is 
majorized  by  p^^.  Ihus  < t(p^)  Ibr  all  Bohur  convex  functions 

sfid  in  particular  symmetric  norm,  ||»||, 

on  . 

V 

Proof.  The  majorlzatlon  of  s^  by  follovs  from  lemma  2 with  n - m 
and  ^ ■>  yit}''*  The  other  statements  are  oonsequenoes  of  majorlzatlon 
(see  lemma  1 and  our  remarks  on  Sohur  convex  functions). 

Remark  1.  Consider  ■ Xj^hYj^  (mod  m)  1 - 1,2,...,N,  vAiere  X^^  and 
Y^  both  assume  values  In  yti.  » (0,1,..., m-l}.  In  this  case  X and  Y 
plsy  aymmetrlo  roles.  It  follows  from  oorollsry  1 that  If  q,^  denotes 
the  distribution  of  Y^  > 

Thus  Sohur  convex  functions.  Also 

note  that  this  conclusion  applies  to  a^  subset  A of  the  index  set. 

Thus  for  all  K < N,  all  k dimensional  marginal  distributions  of  Z 
are  as  least  as  \mlform.  In  the  sense  we  described,  as  are  the  corresponding 
distributions  of  X and  Y. 


Reittttrlt  2«  If  TyX  is  of  the  form  (T^  Xj^#p«.>Ty  Xjj)  ■vAiere  eaoh 
Ty  X4  is  a mixture  of  1-1  onto  transformations  and  X and  Y arc 
Independent,  then  the  conclusion  of  corollary  1 will  hold  for  all  A. 

In  addition  if  wa  have  an  m X m matrix  3 with  rows  labeled  0,.*.,m-l 
and  oolujnni  0,«..|m-l,  with  eadh  row  and  column  containing  each  of  the 
numbers  0,t..,m-l  exactly  once  (an  m x m Latin  »4uare)  then  defining 
B(X^,Yj^)  leads  to  ir(Sj^)  < mln(i|f(p^),\|r(4A) ) for  all  A.  Ihe 
additive  generator,  2^  ■ (“o4  “)i  of  fona* 

Remarh  We  briefly  consider  a generator  proposed  by  Maolarin  and 
Marsaglla  [11],  and  discussed  In  Knuth  [8],  Pi  30-31t  Knuth  remarks 
that  the  method  produces  sequences  with  excellent  randomness  properties 
and  Is  quite  efficient  In  terms  of  computer  time  usage.  Under  this  method 
tile  first  k elements  of  X are  used  to  form  a table.  W*  observe  Y^ 
^loh  tells  us  which  element  of  the  table  to  choose  as  We  replace 

this  element  by  process  Is  then  repeatedly  applied  to 

generate  the  string.  Suppose  that  a string  of  n numbers  Z^,...,Z|^, 
is  generated  by  this  method.  We  artificially  enlarge  this  set  to  size 
nH-k  by  setting  equal  to  the  entry  Which  sits  In  the  1-th  place 

In  the  table  after  the  string  of  n numbers  has  been  generated.  The 
new  string  (Zjl*  * * '^^n+k^  ^ random  permutation  of  (Xj^*** 

Induced  by  Y.  Since  a permutation  of  coordinates  Is  a 1-1  onto  trans- 
formation, theorem  1 applies.  Thus  s,  the  distribution 

of  least  u uniform  in  our  sense  as  Is  that  of 

(Xj^,...,Xjj^jj). 

In  general,  Improving  the  uniformity  of  a Joint  distribution  does  not 
necessarily  Improve  the  uniformity  of  marginals.  For  example  let 


138 


p(0,0)  « p(l,0)  = .1  and  p(0,l)  =,  p(i,i)  „ .4,  i>p(y=o)  « Pr(Y=,i)  « .5^ 

(Zj^^Zg)  = T^(X^X2).  “ •!» 

■ (1,1)  ■ .4  and  a(l,o)  - s(o,i)  - .25.  men  e is  majorized  by  p 
«nd  the  Joint  diatrlbution  of  (Zj_,Zg)  la  more  uniform  on  tO,l)x{o,l) 

than  that  of  (Xj^,Xg).  Nevertheleas  Xj_  la  perfectly  uniformly  distributed 
vihlle  is  not. 


Remarh  4»  In  theorem  1 we  show  that  t(*.)  < '|f(p.)  for  all  Sohur  convex 

A A 

The  Sohur  convex  functions  of  greatest  interest  to  us  are  distances 

from  under  symmetric  norms.  There  are  other  relevant  Sohur  functions 

\*iloh  arise  from  information  theory  considerations.  If  a is  a probability 

distribution  over  7)L^  then  g(a,rj^)  - ^ a(a)log(m^a(o«)),  the  Khllbaok- 

a c7>^ 

LeJbler  Information  number  for  dlsorlmlnatlng  between  a and  r^  when  a 

Iv 

is  true,  is  Sdhur  oonvex|  g(a,rj^)  > 0 with  equality  if  and  only  if  a » rj^, 

and  in  interesting  ways  can  be  Intei'preted  as  a measure  of  discrepancy 

between  a and  r.  (Kullbaok  [10] ).  similarly  g(r.  ,a)  « E ^ n)"^log(m“Va(a)), 

* a e 

the  Mullback-Lelbler  information  number  for  discriminating  between  a and  r. 

A 

vhen  rj^  is  true,  is  Schur  convex,  as  is  g(a,rjj)  + g(rj^,a),  the  divergence 
between  a and  rj^.  Substituting  these  Sohur  convex  functions  into  the 
inequality  < t(Pji^),  derived  in  corollary  1,  strengthens  the  assertion 

that  s is  as  least  as  uniform  as  p. 


4.  Combining  several  generators. 

Suppose  we  have  a sequence  of  Independent  random  vectors  X,,X„,...,X 

-fl' 

We  combine  X^^^  and  Xg  to  form  a vector  then  combine  Zg  and 

3^  to  form  ^ etc.  Assume  that  at  each  stage  the  transformation  is 
of  the  form  =■  a mixture  of  1-1  transformations  of  TTV^ 
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i' 

I 
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r 

I 


onto  ‘fti  . Represent  the  transition  from  stage  n-1  to  stage  n by 

the  matrix  where  a,p  e 

Define  a . to  be  the  distribution  of  Z Then  s , »P  - a . 

n>w  n-ijA  n n^A 

*n,A  «»dorlaed  by  theorem  1}  thus  by  lemma  1 

is  doubly  stoohautio.  The  prooese  (z„  .,n-l,S,...)  is  thus  a non  time 

rk 

homogeneous  doubly  stochastlo  Markov  chain  on  the  state  space  Also 

assume  that  min  ^ > A > 0 for  all  n.  Define  > max 

P Ot 

and  m^^  « min  s^(a).  We  will  show  that  Mj^-m^^  < (l-m^fik)*'  which  implies 

that  max|s  (a)-m"^|  goes  to  zero  at  a geometric  rate.  The  method 
a " 

employed  below  is  well  known  in  the  theory  of  Markov  bhalns.  Now! 


1 

i 


\ ' 


“*  (2)'  »y  lt.r.tlon 

v*iloh  prove  the  resvdt. 

Under  the  weaker  condition  E * - « ve  get  Hm  (M -m  ) n o but 

XI  CO  ” ^ 

the  convergence  need  not  be  geometric.  The  condition  J|  « co  is  not 

necessary  for  convergence  of  M -m  to  zero  (and  thus  of  s to  r ) 

^ ^ Xi^  A It  ^ " 

For  example  If  ^ - m'*  for  any  1 then  . Pj^  for  all  n > l. 


Hfi  .mt.'.n 
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SIMPLIFIED  POINT  AND  INTERVAL  ESTIMATION 


FOR  REMOVAL  TRAPPING 

+ 

Andrew  P.  Soma 


Abstract 

A regression  technique,  based  on  the  limiting  normal 
distribution  of  the  multinomial,  is  given  for  point  and 
interval  estimation  of  the  parameters  in  the  removal 
trapping  method  of  determining  animal  and  insect  popula- 
tions. Pooling  is  described  for  using  the  method  even 
when  the  individual  catches  are  small  and  a simulation 
approach  to  the  calculation  of  bias  is  described.  Two 
examples  of  estimating  spider  populations  are  given. 
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1.  Introduction 

A thorough  dlscusBlon  of  the  removal  trapping  method  of 
eatimatiiig  animal  and  Inseot  populations,  together  with  limita- 
tions, is  given  in  C73,  pp.  182-6.  It  is  pointed  out  in  [5]  that 
this  method  is  particularly  suited  for  insect  populations. 

Briefly,  there  are  assumed  to  be  m organisms  in  some  fixed  area, 
k trapping  or  sweeping  periods,  k > 2,  ahd  each  organism  is 
assumed  to  have  a constant  probability  p of  being  captured  in  any 
of  the  k periods.  Independent  of  the  other  organisms  (the  organ- 
isms are  not  released  when  captured).  If  the  trapping  probability 
is  p,  0 < p < 1,  then,  as  pointed  out  by  Moran  [6],  p.  308,  the 
Joint  density  of  the  n^,  1 < 1 < k,  the  number  of  organisms 
trapped  in  each  of  the  periods,  is 


PCn^-Sj,  ! s^<m,  l<l<k3  ■ ^ — U p*^(p(l-p ) )"^. . . 

k 


. (p(l-p)^-^)''^(l-p)^)  ^ 


k k k 

la.  I (1-1  )8,+k(m-X  8.  ) 

Ni-p)^ 


)l 


The  above  is  seen  to  be  s znult Inomial  distribution » with  k-*-! 

categories,  and  parameters  m and  p.  • p(l-p)^'‘^i  1 < i < k,  and 

k k ^ 

p.  ■ (1-p)^  "I-fPi  p(l-p)^~^.  It  ic  docired  to 

* ^ 1^1 

estimate  m and  p and  give  asymptotically  exact  confidence  intervals 
A Bayesian  approach  has  been  considered  by  Carle  and  Strub  [2]. 

Here  an  attempt  will  be  made  to  remedy  some  of  the  problems  in  [6] 


r<  i isswims’iiart  Mudlt'  ,< 


. . 


In  which  a method  based  on  maximum  likelihood  is  proposed,  which 
is  elaborated  upon  by  Zippin  [8],  In  addition  to  replacing  m!  by 
Stirling's  approximation,  the  effoct  of  which  is  not  clear,  both 
Moran  [63  and  Zlppln  C83  state  that  the  usual  regularity  condi- 
tions for  the  Joint  asymptotic  normality  of  the  maximum  likelihood 
estimators  are  not  satisfied  In  this  case  (In  addition  to  other 
assumptions,  It  Is  assumed  that  the  parameters  being  estimated,  m 
and  p,  remain  constant,  which  Is  not  true  here,  since  the 
asymptotic  behavior  Is  for  fixed  p as  m «),  and  then  they  pro- 
ceed in  the  hope  that  somehow  a Justification  may  be  produced 
without  giving  it.  Further,  even  if  these  difficulties  are 
neglected,  the  estimating  equations  are  either  Implicit,  requiring 
iteration,  or  after  some  approximations,  require  charts.  Here  a 
theoretically  Justifiable  Intuitive  method  is  discussed,  based  on 
the  limiting  distribution  of  the  multinomial,  which  gives  the 
estimates  explicitly  as  functions  of  nj^,.,.,nj^. 

The  author  was  introduced  to  this  problem  by  Joan  Jess  of 
the  Milwaukee  Public  Museum,  who,  as  n part  of  her  Master's  thesis 
wanted  to  estimate  spider  populations.  She  found  the  existing 
literature  somewhat  complloated  and  confusing  and  wondered  whether 
there  was  a simpler  and  more  Intuitive  approach  available.  The 
subsequent  sections  are  an  attempt  to  do  this. 

2.  The  Modified  Moment  Estimates 
for  Two  Tropping  Perlodn 

Sometimes  the  n^  are  so  small  as  to  oast  doubt  on  the  validity 
of  the  asymptotic  method  to  bo  described.  Alternatively,  it  may 
be  desired  to  produ  e quick  and  simple  estimates  and  to  determine 

US 


■,,■8 
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i 


■I! 

'I 

1 


’'li 


I I 


■i 


r ' 

k.. 


whether  biases  exist  (this  will  be  discussed  later).  With  this 
objective  in  mind,  we  will  assume  that  there  are  2t  trapping 
periods  and  divide  them  into  two  groups  consisting  of  the  first  t 
and  second  t.  Then  the  assumptions  of  the  removal  method  are  met 
with  2 periods^  n^^  ■ d'  caught  in  first  t periods,  n^  ■ # caught 
in  second  t,  and 

P - 1 , (2.1) 

where  p^  is  the  original  trapping  probability.  Hence  eetlmates 

and  confidence  Intervals  for  p can  be  immediately  converted  to 

those  for  p and  this  will  be  done  after  obtaining  estimates  for 
0 

m and  p. 

Note  that  En^  ■ mp  and  En^^mpd-p),  hence  a ratio  estimate  q 
of  q ■ 1-p  la 


q 


V"! 


and  a moment  estimate  m of  m is 

‘n. 


m 


"l""2 


1-q 


(2.2) 


(2.3) 


It  is  readily  seen  that  il'  n^  and  n^  are  replaced  by  their 
expected  values  in  q and  m,  q and  m are  obtained  (E(n^-''n2}  ■ 
mp-*-mp(  1-p ) ■ m(l-q  )),  hence  the  name  "modified  moment  estimates". 
Let  p be  the  natural  estimator  of  p,  p > 1-q  here  and  throughout. 

The  limiting  distribution  of  p and  q will  be  now  obtained. 

The  reader  who  is  not  mathematically  Inclined  oan  Jump  to  the 
results,  but  it  must  be  pointed  out  that  the  derivations  are 
straightforward  and  oan  be  checked  by  anyone  with  a modest  back- 
ground in  statistics. 
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Consider  the  problem  of  asymptotloally  exact  oonfldenoe  inter- 
vals — a reasonable  assumption  is  that  p and  k stay  constant  and 
m Note  that  in  [2],  p.  626,  It  is  required  that  also  k-**. 

For  later  use,  instead  of  two  trapping  periods,  k will  be 
considered  and  then  speolallzad  to  two,  The  asymptotic  distribu- 
tions of  p and  m will  be  obtained  by  using  two  results  - the  Joint 
asymptotic  normality  of  n ■ (n^^,  . . . ,nj^ ) and  a result  given  In 
Anderson  Cl],  pp.  76-7.  It  is  well  known  that  as  (p^,>.,,p^) 
stays  constant  and  m « for  a multinomial  distribution. 


n^-mpj 


.(mPjtii ) 


ISlSlc 


^ N(6,R),  (2.4) 


W 

( 'U"  means  convergence  in  distribution),  where  5 la  a k-tupls  of 
O's,  6 ■ (0,0,. ,.,0),  R ■ CPjj],  ■ 1 and  for  1 J , 

^IJ  “ (fl*«»  •♦8*  Johnson  and  Kota  [4],  p.  2S4 ) - 

recall  that  here  pj  • p(l-p)^“^,  l<l<k  (also  for  notatlonal  con- 
venience always  p^,  " P,  ^ ~ ^ hence  it  suffices  to 

keep  p constant.  The  result  cited  in  Anderson  is:  Let  f(i()  be  a 
function  of  X ■ (x^,...,Xj^)  with  continuous  first  and  second 

derivatives  existing  in  a neighborhood  of  x ■ S,  b ■ ( b^^ , , . . ,bjj ) 

^ w 

a fixed  vector,  and  suppose  (U(n)-b)  ■*  N(3,T).  Then 

/n  [f(U(n))-f(b  )]  I N(0,'>gT*i)  , (2.5) 

where  ■ (|^»  • • • • I"  that  follows,  b ■ C ( P^/q^  )*,... , 

1 ^ ^ h 1 1 

(pjf/qj^)  ],  and  U(  n ) ■ ( n^/m(pjqj^ )’,...  ,nj^/ra(  Pj^qj^  )*) . Let  us  now 

assume  that  k ■ 2 and  toko  f(x)  to  be 


f(x) 


(2.6) 
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Note  that  for  k«2,  f(b) 


■ --I  the  estimator  of  q.  Using  (2.5), 

[^  _ q]  ^ N(0,q  (^))  - N(0,op  , 

■ inoa 

b 

(l^).  ■ qC(  1-pq )/(pq )3*  , 

‘'*2  b 

and  R ■ o*.  Therefore  also 
b b P 

^ (p-p)  ^ N(0,Op)  , (2.7) 

or,  equivalently,  p-p  la  aaymptotloally  N(0,Op/m).  Using  the  earns 
technique  on  in,  let 


,(i)  . 

1 - f(x)^ 


with  f ( X ) given  by  (2.6).  It  can  be  seen  that  for  k"2, 
g(U(n))  ■ m/m  and  C(b)»l.  Then 


' ii  U ■ t 

'**1  b 1-q^ 

'“Sx-  V TTS 


‘2  b 


and 


and  so  by  ( 2.2  ), 


(2.8) 


/15(§-  l)I»(0,o;) 


I 





), 


or  equivalently  m-m  is  asymptotloally  N(0,nio^).  Since  p ■»  p and 

2 2 2 2 

fn  ■*■  n in  probability,  o and  o„  converge  to  and  o„,  respectively, 
in  probability  (o^  and  are  obtained  from  and  o^  by  replacing 
p by  p and  q by  q),  and  sc  the  limiting  distribution  of  both 
(p-p)/(Op/m)^  and  ( m-m )/( m )*  la  tha  atandardiaad  normal#  Henoa 
asymptotically  exact  (marginal)  l~a  oonfldenoe  Intervals  for  p 
and  m are 


P * V2 


(2.9) 


and 


mis 


a/2 


(« 


(2.10) 


Where  la  the  upper  lOOo'^h  percentile  of  the  atandardleed  normal. 

Simulation  results  .indicate  that  for  p the  asymptotic  dis- 
tribution is  attained  faster  than  for  m.  An  alternative  approach 
to  obtaining  accurate  99!$  confidence  Intervals  for  m (coverage 
probability  close  to  but  bigger  than  .99  even  when  m is  small)  Is 
to  use  an  actual  standard  deviation  obtained  by  simulation.  In 
this  approach  p and  m are  regarded  as  the  true  values  and  a large 
number  (say  1000)  of  samples  is  drawn  and  the  sample  standard 
deviation  and  bias  calculated.  This  Is,  in  fact,  recommended  In 
any  case,  since  the  comparison  of  the  two  standard  deviations  (the 
limiting  and  the  simulation)  will  Indicate  whether  It  is  safe  to 
use  the  asymptotic  theory.  In  addition,  m can  be  corrected  for 
bias.  This  point  will  also  be  discussed  in  subsequent  sections. 

In  order  to  convert  the  estimate  and  confidence  interval  for 
p to  Pp,  note  that  p ■ 1-(1-Pq)*  and  hence  q^^  » q^^^.  Also,  If 
the  confidence  Interval  for  p is  (a,b),  then  the  corresponding 
interval  for  p^  Is  ( l-(  1-a  )^'^^,  l-(  1-b  )^'^*  ) . 
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The  ilaok  of  fit  dloouaslon  will  be  given  after  the  general 
eatlmatee  are  obtained,  sinoe  it  will  be  Been  that  the  p and  tn 
here  are  a speolal  oase  when  k>2. 

3.  Regreaiioti  Eatimatea  for  Arbitrary  k 

While  the  method  diaouaaed  in  2.  la  uaaful  when  the  oatehea 
in  tha  individual  parioda  are  amall,  it  ia  often  daairable  to  uaa 
the  original  data  if,  a.g.,  all  m 2;  where  p^  > and  p and 

m are  auitabla  eatimatea  of  p and  m.  An  aatlmata  of  p,  tha 
ragraaalon  eatimato,  oan  be  obtained  aa  followa.  It  la  euggeated 
in  C5]  that  a almplt  graphloal  ohaok  of  the  validity  of  aaBumptiona 
ia  to  plot  n^  agalnat  1 on  aemi-log  paper  and  oheok  whether  the 
plot  approxlmatea  a atralght  line.  It  la  pointed  out  in  [6]  that 
the  log  Enj^  ■ log  ■ log  mp(l-p)  ’ lie  on  the  atralght  line 

log  " 1 log(  1-p } - log(  l-p)  + logp  + login,  (3.1) 

but  thla  method  la  then  dlamiaaed  by  aaying  that  the  uaual  aaaump- 
tiona  of  ragraaalon  theory  are  not  aatiafied,  Here  a different 
approach  la  taken  - namely,  the  point  eatlmator  of  p euggeated  by 
regreaaion  theory  ia  uaed  but  then,  In  place  of  the  ueual  regrea- 
aion  theory,  the  limiting  dlatributlon  la  obtained  from  (2.9), 

The  regreaaion  equation  auggeated  by  (3.1)  la 

log  n^  ■ 10  + a + Cj  , lsl<k  , 

where  0 ■ log(l-p)  (to  any  baae),  a a oonatont,  and  the  error 
term  which  will  be  of  no  intereat  here.  The  leaat  aquarea  eatlm- 
ate  8 of  0 la 

0 ■ jdog  rij  )(1  - ^S^)/J(l  - . 

ISO 
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Slnoe  I • (k)(k+l)(2k+l)/6, 

1 

B - ydoe  n^)(l -^)/(k(k^-l)/12)  . 
Thtn  th«  Qorrtipondln;  •itlnifttor  q > 1-p  of  q ■ 1-p  is 

- , - S °i 

q-l-p-nn.  , 

1 ^ 


(3.2) 


whori  0^  " ( l-(  k'*-l  )/2 )/( k(  k^-1  )/12  ).  Not#  that  raplaoainant  of 
by  En^  In  (3.2)  glvaa  q.  In  ordar  to  ahow  that  (3.2)  la  oonola- 
tant  for  p and  to  obtain  Ita  aaymptotlc  varianoa,  it  la  Juat  aa 
aaay  to  oonaldar  ganaral  aatlmataa  of  l>p  of  tha  form  (3.2)  with 
tha  Qj,  arbitrary  and  to  datarmlna  tha  oondltlono  on  naadad  for 
oonalatanoy.  Lat 


h(x)  ■ n (xj(Piqi )* ) ^ 


(3.3) 


Than,  ualng  (2.5)  and  pj^  ■ p(l-p) 


k k k k 

k 0,  J®!  k 0^  3*  0.  f°l  Z°1  I(i-l)oj 

/i  (n  n.  */in^  - n p,  ^)  ■ (n  n.  - p^  (1-p)^  ). 

1 ' 1 ^ 1 ^ 

Tharafora  a auffialant  oondltlon  for  oonalatanoy  la 


I "1  ■ 


and  I iOj  ■ 1 


and  In  thla  oaaa 


k 0, 


(3.4) 


/m  (n  n.  -d-p))  N(0,o‘“)  , 

1 1 P 

whara  o^  la  datermlnad  In  tha  uaual  way  from  (3.3)  ualng  (2,5). 

It  la  noted  that  for  Oj  ■ ( l-( k+1  )/2  )/( k( k^-1  )/12  ) (3.4)  la  aatla* 
flad,  alnoa  olaarly  | o^  ■ 0 and,  letting  o ■ k(k^“l)/12, 


1 1«,  . i i(i-i;i)/o 

•«  X • 


■(k)(li+l)(2k+l)  k(k+l)^' 

6 


/o  ■ 1 


So»  for  any  oholoa  of  o.  aatlafying  (3.4)i  o^  o 

1 P D b 

♦S  . (^ .nd  •!».. 

k 1 0, 


‘k  b 


<V*l>f(*i(Pl»l!‘>  ‘ 


«•  hava  that 


(3.5) 


<^>5  ■ "itli/Pi)*.!  . 

Tharafora,  aftar  aoma  abort  algabra^ 

oj  ■ ! oJ/pj  . 

A aatlafaotory  aatlmator  of  n la  obtalnad  by  tha  aama  argumant  aa 
for  tha  nomant  aotlmatort  namaly, 


(3.6) 


A 

m 


I n,/(l. 


''k^ 

q ) > 


(3.7) 


whara  q la  glvan  by  (3.2 )»  with  tha  Oj  arbitrary  and  aatlafylng 
(3.4).  It  la  aaen  that  raplaoainant  of  random  varlablaa  by  their 
axpaatatlona  in  (3.6}  givea  m,  Aa  before,  let 

k k Oj 


g(x)  - J x^/pjq^ /(I  - (n  (Xj/pJ^) 


'l.k 


r) 


(3.8) 


Than,  by  (2.5), 


/s  (2-1)  * "(O.oJ)  . 


wh.r.  oj  . *5  . (|I- |1-)  .„d 


k b 


(p^q^  )*  ♦ kOj(q^/Pj  )*(  1-p)^ 


/ ^ 


Using  (3>9)>  after  some  simple  algebra, 


v2„k  k c,‘ 

I ^ 

1-4  1 ^1 


(3.10) 


The  regression  estimates  q and  m are  obtained  from  (3.2)  and 
(3.7)  by  choosing  ti^  ■ ( l-(  k+l  )/2 )/(  k(  k^-l  )/12  ),  and  their  var- 
lanoes  from  (3.6)  and  (3.10).  It  then  follows,  exactly  as  In  2., 
that  asymptotically  exact  1-a  confidence  Intervals  for  p and  in  are 


P * V2 


(3.11) 


” * V2  > 


(3.12) 


2 2 2 2 ^ 

where  aZ  and  are  obtained  from  o~  and  by  replacing  p by  p. 
p m p xR 

It  Is  interesting  to  note  that  If  ka2,  the  estimates  given 
here  coincide  with  those  In  2. 

It  should  also  be  pointed  out  that  there  Is  an  alternate  way 
of  estimating  m from  (3.1)>  namely  by  equating  the  estimate  of  the 
constant  term  with  what  it  estimates.  Without  going  Into  the 
details,  the  resulting  estimator  m Is 

( S 

” = -r*-rnriT. — (3.13) 


and  the  asymptotic  variance  of  of  (? . 1)  is 

in  ^ 


k , p, 

I — r + 0. 
i Pi  i 


(3.14) 


where  correspond  to  the  regression  estimator  of  q.  The 
estimator  given  by  (3.13)  has  an  Interesting  form  - it  Is  the 
ratio  of  the  geometric  mean  of  the  n^  to  the  geometric  mean  of 


A 


r- • “•.V;“k*'.-V-V--.'V-:'--' : •■  '•r'-’ 


the  pj.  If  random  variables  are  replaced  by  their  expectations 
In  (3.13)»  m results.  The  asymptotic  variance  (3.14)  Is  very 
close  to  (3>10)  with  the  regression  o^.  Simulation  studies  Indi- 
cate, however,  that  (3>13)  has  a substantially  bigger  bias  and 
sample  variance  than  (3.7),  which  is  the  preferred  estimator. 


4.  Lack  of  Fit 


Having  estimated  the  parameters,  using  the  regression  method 
as  In  3«  (recall  2.  Is  a special  case),  It  Is  natural  to  ask  how 
good  the  fit  Is.  The  customary  statistic  used  to  test  the  ade- 
quacy of  the  model  Is  Z ■ J )^/( inpj  ) , where  p^  ■ p(  1-p  . 

It  is  not  at  all  clear.  In  this  or,  of  course,  the  maximum  llkell- 

p 

hood  case,  that  Z has  an  asymptotic  (p  and  k fixed,  m ■*■  »)  x 

distribution  with  k-2  degrees  of  freedom  (d.f.),  since  the  usual 

regularity  conditions  (see  C3],  pp.  900-1,  906)  are  not  satisfied, 

The  empirical  approach  given  here  consists  of  using  (2.9)  to  obtain 

the  expected  value  of  the  limiting  distribution  of  Z and  then  to 

fit  a X distribution  (as  Is  done  with  good  results  in  fitting 

2 

the  distribution  of  sums  of  ;<  random  variables)  by  estimating 
the  d.f.  from  the  parameter  estimates.  The  observed  value  of  Z 
is  then  compared  to  the  upper  lOOo^^  percentile  of  the  fitted  x^ 
(using  Interpolation  of  the  d.f.,  since  in  general  the  fitted  d.f. 
will  not  be  Integral).  Specifically,  consider 


f^(x)  - Xj(pjqj)*  - g(x)(l-f(x))(f(x))^"^  , 

where  f(x)  and  g( x ) are  given  by  (3.3)  and  (3.8),  respectively. 
Note  that  fj(b)  ■ 0 and  thus  from  (2.9), 
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(4.1) 


L 


f^(  n^/(m{  p^q^  )*),.. . ) ) 


» /m  ((n.-nlpq^"^)/m)  ■>  N(0,of)  , 


w 


where  of  ■ ♦jCTOlr, 
1 ID  ID 


^3fj(x)  3fj(5)^ 


'■k  b 


3f.(x) 

Using  (3.5)  and  (3.9)  and  the  chain  rule,  ( ^ ) is 

b 


3f.(J ) 


“ J b b ^ ^ 9*j  h S’tj  b 


(4.2) 


if 


i y j,  and  (p^qj)^  is  added  to  (4.2)  if  i“J.  From  this,  Oj 


can  be  computed.  By  (2.5),  since  (in/m)^  •*•  1 and  pj  p in 
probability,  it  follows  from  (2.5)  and  (4.1)  that 


(n^-mpq^"^  )/(mpj  )^  N(0,Oj/pj) 


and  therefore  the  asymptotic  mean  of  Z ■ ^ ( n . -mpq^“^  )^/( mp.  ) is 

V T 1 1 


y ■ I o^/p^.  Replacing  the  parameters  p^  and  q^^  by  their  estim- 
ates Pj^,  q^  in  y gives  the  estimated  d.f.  of  the  distribution  of 
Z,  and  using  these  d.f.  a cut-off  point  for  the  adequacy  of  fit 


test  can  be  obtained  from  tables.  Even  though  explicit  expressions 

.2 


do  not  appear  practical,  the  o^  and  u are  easily  evaluated  by 
means  of  a short  computer  program. 


5.  Numerical  Examples 


We  illustrate  the  preceding  theory  by  two  data  sets  of 
thomisid  (crab)  spiders  furnished  by  Joan  Jass.  The  first  set  has 
k~5  and  ( n , n^ , , n^ , n j ) ■=  ( 37,29,17,15,12)  and  the  second  k*6  and 
(n^,n2,n^,n^,nj,n^)  ■ (46,29,36,22,26,23).  The  results,  using 
the  regression  estimators,  are  in  Table  1. 


Table  1 


1®*  Data  Set 


Z"**  Data  Set 


17.4 


Teat  of  Fit 


4.06 


It  might  be  gueesed  that  beoause  of  the  small  value  of  p in 

the  2^^  set  there  would  be  a serious  bias  in  the  estimate.  A 

simulation  was  done,  for  both  data  sets,  using  4000  samples  and 

''k 

the  estimates  in  Table  1 as  the  true  values,  oondltlonal  on  1-q 
being  equal  to  or  bigger  than  .001  - while  this  value  is  arbitrary, 
the  bias  was  not  sensitive  to  it.  As  expected,  the  bias  in  p was 
negligible  for  both  sets.  The  bias  in  m was  5.7  for  the  first 
set  and  60  for  the  second.  Thus  while  it  is  satisfactory  to 
subtract  5.7  from  m in  the  first  set  to  obtain  137  as  an  estimate 
and  use  this  in  the  confidence  interval  procedures,  it  is  clearly 
desirable  to  pool  the  intervals  in  the  2^^^  set.  After  this  was 
done,  using  k°2,  the  estimate  of  m was  306,  with  s.d.  66.5,  quite 
close  to  what  would  have  been  obtained  by  subtracting  the  bias 
from  the  original  estimate.  The  actual  sampling  variance  of  m was 
340.1  as  compared  to  the  theoretical  limit  of  88.2.  For  the 
reduced  set  the  two  values  were  212.5  and  66.4,  Indicating  that 


the  confidence  Intervals  based  on  the  limiting  distribution  should 
be  viewed  with  caution.  For  the  first  set  the  sampling  variance 
was  24.6  and  the  limiting  value  17.23> 

Simulation  studies  support  the  view  that  if  the  bias  is  not 
large  (equivalently  If  p is  not  close  to  0),  replacement  of  the 
limiting  variance  by  the  sampling  variance  yields  realistic  (in 
fact,  slightly  conservative)  confidence  intervalSt  even  when 
m is  as  small  as  100. 

The  listings  of  the  two  short  computer  programs  used  for  the 
above  analysis^  the  estimation  and  simulation,  are  available  on 
request  from  the  author. 


6.  Summary  and  Concluding  Remarks 

For  ease  of  use,  we  list  the  estimation  and  confidence 
interval  formulas  in  2.  and  3.  If  k*2,  then 


q - , p ■ 1-q  , m ■ ( n^+ng  )/(l-q‘' ) 


"p  ■ <1  * . 

pm  1-q  p 

and  asymptotic  100(l-a}X  confidence  intervals  for  p and  m are 


‘ V2  ‘p 


•"«  " ‘ ‘a/2  • 


where  ^^^2  100o/2  upper  percentile  of  the  standard  normal. 

If  k>3 

q-nn,  p-l-q,  m-[  n./(l-q’')  , 

1 ^ 1 ^ 

where  c^  ■ ( i - k(  k^-1  )/12  ),  and 


■ 


si . (iVs)  I c^/p.,  4 . fL 

i J-  “ *1 


.2-1:  k c?  “■ 
1 4 I -i- 

1-q  1 J 


and  asymptotlo  lOO(l-0()^  oonfldenoa  Intervals  for  p and  m are 


i * ‘a/2  ‘p  “ * ‘a/2  ‘a  ' 


elth  liefore.  To  oaloulate  the  laok  of  fit  statlstlo  and 

Its  d.f.p  It  Is  simplest  to  use  the  short  computer  program  referred 
to  above. 

In  summaryp  the  purpose  of  this  paper  has  been  to  give  a 
statistically  Justifiable  and  conceptually  and  computationally  simple 
method,  the  regression,  as  an  alternative  to  the  maximum  likelihood 
approach  which  suffers  from  three  deficiencies:  the  standard 
regularity  condition^  for  the  Joint  asymptotic  normality  of  the 
maximum  likelihood  estimators  are  not  satisfied,  the  estimating 
equations  are  either  implicit  or  require  the  use  of  charts,  and 
It  Is  complicated  appearing  to  non-speolallsts . 

The  methods  discussed  here  should  also  be  useful  In  other 


cases  where  the  data  Is  multinomial  and  the  standard  maximum 
likelihood  regularity  conditions  are  not  satisfied. 
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REGRESSION  FOR  MARKOV  BERNOULLI  RANDOM  VARIABLES 
Ednond  K.  iQuelmaan 

US  Army  Combined  Arm*  Combat  Development  Activity 
Fort  Leavenworth-,  Keneae 

1.  INTRODUCTION.  ThB  probim  studied  was  that  of  regression  on  Bernoulli 
random  variables  In  the  case  where  some  of  the  random  variables  were 
dependent.  The  Interest  In  this  case  arose  from  a problem  of  trying  to 
fit  probability  of  hit  curves  to  data  generated  by  repeated  missile 
simulations  performed  at  US  Araiy  Materiel  Systems  Analysis  Activity  using 
tracking  data  from  the  Antitank  Missile  Test  (ATMT).  Kit/miss  data  ware 
generated  one  second  apart.  Because  overlapping  tracking  data  were  used* 
successive  shots  were  dependent.  This  caused  problems  that  seemed  In- 
surmountable until  the  author  became  aware  of  Klotz's  papers  (1)  (2). 

In  these  papers  the  parameters  of  a sequence  of  Bernoulli  dependent 
random  variables  satisfy  the  Markov  chain  property.  In  the  case  of 
successive  shots,  the  assumption  of  Markov  chain  seemed  reasonable  and 
was  used  to  solve  the  problem.  Klotz*s  technique  was  extended  to  the 
regression  problem. 

2.  PRELIMINARIES.  In  the  generated  data  the  following  occurred:  for 
several  different  ranges,  a number  of  gunners  (the  number  was  not  the  same 
for  all  ranges)  fired  a sequence  of  shots  (not  til  the  same  sequence 
length).  The  shots  were  fired  a second  apart.  Let  X(ItJ.R)  be  the  results 
of  the  Ith  shot  of  the  Jth  gunner  at  range  R.  A hit  caused  X to  be  1 and 

a miss  caused  It  to  be  0.  The  notation  that  Is  now  Introduced  Is  that  of 
Klotz  but  modified  to  the  needs  of  the  problem  under  consideration.  The 
first  probability  of  hit  Is: 

P(R)  ■ Pr  - b,  + bjR  + bjR*  Eq  1 

which,  as  shown  In  the  above  equation.  Is  taken  to  be  a second  degree 
polynomial  In  R.  Next,  the  probability  of  a hit  given  that  the  previous 
shot  was  a hit.  Is: 


X(R) 


which  Is  also  taken  as  a second  degree  polynomial  In  R and  which  Is  the 
lower  right  hand  term  In  the  transition  majbrix.  Clearly,  equations  1 and 
2 hold  only  when  the  sequences  are  stationary,  which  was  a reasonable 
assunptlon  for  the  problem  considered.  The  remaining  three  terms  of  the 
transition  matrix  are: 


flA(«) 


P(R1[1  -XOl 


"idR  • ‘l«1.J  JR  • 0 « 


foot")  • » • 'lO^l  • f j *1  JR  ■ ®l  *1-1  JR  • 0 j ‘1  « 

3.  LIKELIHOOD.  Having  tha  abova  maehlnary,  tht  joint  probability  of  tha 
data  It! 


P'*  XlJR  * " 


R jli  j [i  - p(R)] 


1 - X. 


» p„(R)*’i"  ^ PiodO*’^*  “ ■ J"’ 


(1  - X.»)  X.  , .p  (1  - X4fp){l  - X.  R 

Poi(R)  ^ ^ 


whara: 


Nr  ■ numbar  of  gunnart  firing  at  ranga  R 


nji^  ■ numbar  of  ihots  by  tha  Jth  gunnar  at  tha  Rth  ranga. 
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Substituting  rjg»  Sjp»  and  t^j^  as  dascrlbed  In  equations  8 through  10i 
equation  6 becomes: 


R J-1 


[I  - *P(«)  * 


(s«  - r.o) 


"(nio  - 2 - S^B  + t^o) 


P(R)'^J«  . P(R)]  ^ " *JR 


Where: 


"JR 

lig  ^i-1  jR  ^iJR 


®JR  " ilx  ^IJR 


*JR  ■ ^IJR  * *njg  JR 


Eq  10 
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Then  the  likelihood  function  Is: 


N(R) 


" R jJl  “ V - ‘jr]  ^ ^ 


+ (njp  - 1 - 2Sjn  + iTjij  + tj^)  In  (1  - 2P(R)  + x(R)P(R)) 


* - fjR)  InP(R) 


• <"jR  - * - SjR  ♦ ‘ir)  t‘  - «*» 


Eq  11 


Now,  substituting  x(R)  and  Pjj{R)  In  the  likelihood  function  one  has: 


N(R) 


L ■ E E r jp  In  ( s ii.R'' ) 

R J-1  I k-0  R 

♦ [2«jr  - --jr)  - ‘jr]  <>  - X •/) 

♦ ["JR  - ‘ - »jR  ♦ "JR  ♦ *JrJ 


In  (1  - 2 E b-R**  + z b_RR  E e.R*^) 


q«0  ^ k«0 


* <SjR  - t^r)  in  b^RR) 


Eq  12 


-<"jR-*-SjR*  V 
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‘'1ii'l'*ttift'il  ir  vftiMriTiriinifUi'rintininiiii  n i n n i i 


To  find  tht  ffloxlmum  likelihood  estlnetes  of  the  regression  coefficlentst 
partial  derivatives  of  the  likelihood  function  with  respect  to  the  a's  and 
b's  are  required.  These  partlals  are: 

^ . j • *ji>1  *" 

R J-1  -J  j ok  j ‘ , ak 


A ■i.r 

k*0 


("jR  ■ > • * V »" 

2 fl  2 n 2 J 

1-2  E b«R**  + I b^R'^  s e.R** 

q.O  <1  q>o  ^ k-0  * 


Sq  13 


N(R) 

E E 
R j-1 


("jR-»-2SjR*rjR*  V(-2R".R"^E^  ./) 

2 - 2 „ 2 I, 

1 - 2 E b-IT  + E b-R^  E 

q-0  ^ q-0  ^ k-0  * 


Eq  14 


" "".IR^*^  + ^"1R  " ^ " ^JR  * *.1R^ 


E b R** 
q-0  ^ 


1 - E b«R^ 
q-0 


These  expressions  are  set  to  zero  and  solved  for  the  a's  and  b's.  It  Is  clear 
that  the  solutions  must  be  obtained  by  Iterative  methods.  A program  was  written 
to  do  this  using  the  Newton  Raphson  method  (3). 


4.  CONCLUSION.  R«ca11  that  tha  problam  discussad  In  tha  Introduction  was 
tha  problam  of  fitting  probability  of  hit  curvas  to  data  ganeratad  by  ra- 
paatad  misslla  simulations.  Tha  curvas  wara  assumad  to  be  quadratic 
functions  of  R axpreasad  as  follows: 


P(R)  ■ bjj  + bjR  + bjR*  Eq  I 

P^lR)  ■ Rq  ♦ + *2^*  Eq  2 


Hancai  utilixatlon  of  tha  maximum  likelihood  tochniquai  given  by  aquations 
11  through  14  above,  and  subsaouant  solution  by  tha  Newton  Raphson  method, 
provides  tha  values  of  tha  coafflelantSi  a's  and  b'l*  necessary  to  achieving 
a maximum  likelihood  "bast  fit"  of  aquations  1 and  2 to  their  respective 
data  points. 
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ABSTRACT 

Wa  brlafly  ravlaw  tha  usa  of  smoothing  spllnas  and  tha  mathod  of 
ganaralliad  crois  validation  (GCV)  for  smoothing  discrata  noisy  data  from  an 
unknown  but  smooth  ourva.  Than  wa  daicriba  tha  usa  of  *'p1oqua  minca"  or 
Laplaclan  smoothing  spllnas  with  GCV  for  smoothing  discrata  noisy  data  from  an 
unknown  but  smooth  surfaca.  A numarical  algorithm  for  this  (non-trivlall)  computational 
problem  Is  daserlbad.  and  an  axampla  from  a Monta  Carlo  study  Is  prasantad  to 
show  how  tha  mathod  works  on  simulatad  data.  Tha  results  are  axtramaly 
promising.  Soma  design  problems  are  briefly  mentioned.  Soma  conjectures  are 
made  concerning  optimality  properties  of  Laplaclan  smoothing  splines  and 
Laplaclan  hlstospllnas. 
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How  to  Smooth  Curves  and  Surfaces  With  Splines  and  Cross-Validation 


1.  Introduction 

In  the  conference  talk  we  considered  four  problems.  The  first  two  had 
to  do  with  estimating  curves  when  they  are  observed  discreetly  and  with 
error.  The  model  Is 

y^  ■ f(t,|)  + , 1 ■ l,2,...,n 

where  f(t),  t e [0,1]  Is  an  unknown  curve,  only  known  to  be  "smooth" , 

1 ^1  1 •••  and  are  Independent  zero  mean  random  variables 

2 

with  a common  unknown  variance  o . The  {y^}  are  observed.  The  first  problem 
Is?  How  should  f bo  estimated  nonparametrically  from  y ■ lyi,...,y_)'?  The 
second  (or  design)  problem  Is:  How  should  the  points  (t^l  be  chosen  so  that 
the  estimate  of  f Is  as  good  as  possible?  The  third  and  fourth  problems  have 
to  do  with  estimating  surfaces.  The  model  Is 

z,  ■ u(x,.y,)  * . 1-1.2 

where  u(x,y),  (x,y) c some  region  In  the  plane.  Is  only  known  to  be  "smooth". 
(x.|,y.|),  1 " l,2,...,n  are  n points  In  this  region,  the  e.|  are  zero  mean 
Independent  random  variables  with  common  unknown  variance  and 

z ■ (z-j observed.  The  third  problem  is:  How  should  u be 

estimated  nonparametrically  from  z.  The  fourth  (or  design)  problem  Is:  How 
should  the  points  (x.|,y.|},  1 ■ 1 n be  chosen  so  that  the  estimate  of  u 

Is  as  good  as  possible.  We  will  not  discuss  the  design  problems  here.  The 
design  work  mentioned  In  the  talk  has  appeared  In  Athavale  and  Wahba  (197B) 
and  Wahba  (1971,  1974,  1976,  1978c}.  That  work  (and  the  work  of  others, 
mentioned  there)  represents  only  some  first  steps  In  design  problems  for 
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nonparametrlc  curve  and  surface  fitting.  There  are  many  open  problems. 


Very  good  and  relatively  complete  results  for  the  first  (curve  estimation) 
problem  are  available  (Craven  and  Wahba  (1979),  (CM)  and  Golub,  Heath  and 
Wahba  (1977)  (GHW),  and  transportable  code  Is  available  from  at  least  three 
sources  Flelsher,  (1979),  Merz  (1970a),  and  Paihua  (1978).  We  will  briefly 
summarize  those  results,  because  they  will  aid  In  understanding  our  discussion 
of  the  third  problem,  that  Is,  surface  smoothing.  The  remainder  of  this  paper 
will  then  be  devoted  to  the  problem  of  smoothing  of  surface  data  non> 
parametrlcally.  :‘ome  i!ce  theoretical  results  are  available,  and  we  have 
turried  them  Into  a cor.iputer  program  which  delivers  very  pleasing  pictures. 

The  development  of  the  program  1s  the  work  of  Mr.  James  Wendelberger,  and  It 
and  other  results  will  appear  in  his  Ph.D.  thesis. 


2,  Curve  Smoothing 

For  curve  smoothing,  we  recottroend  that  f be  estimated  by  the  solution 
m \ 0'^  the  minimization  problem;  Find  f e W*”  ■ {f;  f,f ' , . . . ^ abs. 

ripIniA 

cont.,  f^™^  e to  minimize 


n 1 

F ^ ^ ^(f^'"^(u))2du  . 


The  first  term  represents  Infidelity  of  f to  the  data,  and  the  second  term 

represents  "roughness"  of  the  solution,  The  parameter  x represents  the 

tradeoff  between  the  two.  m«2  represents  "psychological"  smoothness  (we 

thlnkl)  and  Is  frequently  used,  and  gives  good  results.  We  briefly  discuss 

the  determination  of  m from  the  data  later.  The  solution  Is  known  to 

be  a polynomial  spline  of  degree  2m-l.  The  parameter  X Is  chosen  from  the 

data  by  the  method  of  generalized  cross-validation  (GCV).  GCV  Is  derived 

(k) 

from  CV  ("ordinary"  cross  validation).  CV  goes  as  follows:  Let  fp^,f|^;^ 


t ! 

i ■ 





be  the  solution  of  the  minimization  problem  of  (1)  with  the  kth  data  point 
omitted.  The  value  A will  be  a good  choice  If  comes  close,  on 

the  average,  to  yj^.  We  measure  this  by  the  "ordinary"  cross  validation 
function  Vq(a)  ■ vj^Cx), 

For  fixed  m the  parameter  x Is  chosen  by  minimizing  Vq(x).  For  technical 
reasons  Involving  convergence  proofs,  we  replace  Vq(x)  by  the  generalized 
cross  validation  function 

where  the  (W|^(x}}  are  certain  weights  to  reflect  unequally  spaced  data,  end 
effects,  etc.  Details  are  given  In  CW  and  6HU.  It  turns  out  that  V(X)  Is 
much  easier  to  compute  than  Vg(X),  and  V{x)  has  the  representation 

Trace(I-A(x)))^ 


where  A{\)  Is  the  nxn  matrix  which  Is  uniquely  determined  by 


^,m,X<^l) 


'n,m,X^^nV 


A(X)v  . 


Fleasing  results  have  been  obtained  using  smoothing  splines  with  6CV  In  both 
Monte  Carlo  studies  ..nd  various  applications,  Benedetti  (1977),  CW,  GHW,  Merz  (1978a, 
1978b),  Stutzle  (1977),  Utreras  (1978a),  Welch  (1979).  These  results  are  not 
surprising  in  the  light  of  the  following  theoretical  result  (CW,  GHW).  Let 
R(x)  "4  I (^«  « i(t,)-f(tj)^.  R(x)  Is  the  "true  mean  square  error" 

1*1  ngnigA  1 1 

averaged  over  the  data  points.  Before  data  are  observed  both  R(x)  and  V(x) 
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can  be  considered  functions  of  the  unknown  f and  random  functions  of  the 
{e^}.  Let  \*  be  the  mlnimizer  of  ER(x)  and  X be  the  minimi zer  of  EV(x). 

We  have  under  rather  general  circumstances  (see  CW,  GHW) 

Jim  ^ 

Thus  (very  loosely),  the  mean  square  error  with  the  estimated  x tends  to  the 

minimum  mean  square  error  achleveable  with  any  x.  Let  x be  the  mlnimizer  of 

V{x).  Numerical  results  based  on  Monte  Carlo  studies  with  m-2  reported  In  CW, 

with  n-50  and  equally  spaced  data  points,  show  the  achieved  Inefficiency 

R(x)/m1n  R(xj  In  the  range  1.01  to  1.42. 

X 

Some  numerical  experiments  to  assess  the  effectiveness  of  choosing  m 

by  GCV  have  been  done.  (Lucas,  1978).  One  obtains  v'"(x)  for  each  m and 

minimizes  V (x)  over  m.  The  results  Indicate  that  this  procedure  does  a 

good  job  of  picking  out  the  m and  x which  minimize  R(x)  > r'''(x),  and  that 

there  are  classes  of  f's  for  which  it  Is  worthwhile  to  do  this,  that  Is, 

min  r'’’(X)  Is  usefully  less  than  min  R^(x)  for  some  mfZ.  Efficient 
X 

transportable  code  Is  not  presently  available,  however.  Depending  on  f, 
reduction  In  Inefficiency  of  several  percent  can  be  obtained. 


3.  Surface  Smoothing 

We  now  turn  to  the  third  problem,  that  of  recovering  smooth  surfaces. 

We  recommend  that  u be  estimated  by  the  solution  u.  „ , of  the  minimization 

n,m,x 

problem:  Find  u c H (an  appropriate  space,  to  be  described)  to  minimize 

n j l“(x^.y^)-z^)2  + X I If  (y)(  rVj^^  • < 

J"1  J-1  3x  ay 

and  that  X (and  possibly  m)  be  estimated  by  GCV.  We  now  describe  how  to  do 
this.  For  mathematical  convenience  the  limits  on  the  double  Integral  In 
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(3)  are  taken  to  be  -•  and  ..  H 1s  taken  as  H “ h”’(R^)  « {u:  u e p, 

j»ai. . 

— j 'n.-j  E Lp(Ro)i  J ■ 0,1,. ..,m  . (p'is  the  dual  of  the  Schwartz  space  P of 

5xJ  ^ ‘ 

Infinitely  differentiable  functions  with  compact  support,  this  need  not  concern 
us  here,  see  Meinguet  (1978,  1979),  Schwartz  (1966).) 


Theorem;  Let  t^  ■ (x^.y^),  t ■ {x,y)  and  |t-t^|  ■ ((x-x^)^+(y-y^)^)^^^. 

Let  m i 2 and  n >.  M ■ ("2b • The  solution  to  the  problem:  Find 

u c K to  minimize 

1 I (U(t,).z,f  t J /;  ? (?)  ( }\j)^  dxdy  . (4) 

" 1-1  ' ' j-0  J sx’ay"'  -J 

1$  given  by 

n M 

' jJ,  ‘J  • <5) 

where 

E„(s.t)  - e^|s-tl2"*’2  iog(s-t|  , . (2^'"‘’irC(m-l l!]^)-'' 

()>^(t)  ■ x“y®  V ■ 1,2,...,M 

where  a,  6 run  over  all  the  M combinations  of  non-negative  Integers  with 
a+B  < m-1,  provided  the  nxM  matrix  T with  1v^^  entry  ♦^(t^)  Is  of  rank  M. 

The  coefficients  c ■ (c^,...,c^)'  and  d - (dj ,. . . ,d|^)' are  determined  by 

(K+pI)  c + Td  ■ z (6) 

T'c  - 0 (7) 

where  K Is  the  nxn  matrix  with  Jk^^  entry  E^(tjit|^),  and  p ■ nx. 

This  theorem  Is  essentially  due  to  Duchon  (1976a,  1976b).  Meinguet  (1978)) 
has  also  proved  very  similar  results  In  a reproducing  kernel  Hilbert  space 
setting.  For  completeness,  In  the  Appendix  we  outline  a proof  which  roughly 
follows  Meinguet's  argument.  By  putting  the  minimization  problems  of  (1)  and 
(4)  In  a reproducing  kernel  Hilbert  space  setting,  and  can  be 


a.<.'3-rJi^'-'*»Tae'.'*.-*»«-r  * 


shown  to  be  Bayes  estimates  with  a certain  (partially  improper)  prior  on  f 
or  Ui  see  Uahba  (1978a). 

4.  An  Algorithm  for  Computation  of  the  Smoothing  Surface 
We  now  want  to  compute  u„  ^ , efficiently,  and  choose  x (and  possibly  m) 

n iiiif  A 

by  GCV.  Our  algorithm  below  has  benefited  from  the  algorithmic  work  of 
Paihua  (1978).  However  it  is  different  and  seems  especially  well  adapted  to 
determining  the  generalized  cross  validation  function  V(x)  for  this  case. 

We  next  derive  the  equations  behind  our  computational  approach. 

Let  R be  any  nx(n-M)  dimensional  matrix  of  rank  n-M  satisfying  R'T  « 0(n-M)xM’ 
Subscripts  indicate  the  dimensions  of  the  subscripted  matrix.  Since  T'c  > 0, 
we  have 


c - Ry  (8) 

for  Y « unique  n-M  dimensional  vector.  Left  multiplying  (6)  by  R*  and  substituting 
(8)  into  (6)  gives 

R'(K+pI)Ry  - R'z  (9) 

Y ■ (R'(K+pI)R)‘^R'z  (10) 

c - R(R'KR+pR'R)’^R'z  . (11) 


The  vector  d is  then  given  by  d - (T'T)”^' (z-Kc) , obtained  by  left 

multiplying  (6)  by  T',  To  estimate  x (equivalently  p)  by  GCV,  we  want  to 

choose  X to  minimize 

i||(I-A(X))z||2 

V(X)  - . 

(1  Trace(I-A(X)))^ 

where  A(x)  is  the  nxn  matrix  determined  by 


173 


and  define  r and  6 by 
B ■ r6r  , 

where  r and  6 are  the  orthogonal  and  diagonal  matrices  In  the  eigenvalue 
decomposition  of  B.  Then  the  right  hand  side  of  (14)  becomes 

U(r4r'+p)"^U' 

■ l)r'(«+el)‘'ru' 


where  b^.... «b^_|y|  are  the  diagonal  entries  In  6 (I.e.  the  eigenvalues  of  B). 
Given  ur',  {b^}  we  compute 


c ■ Ur' 


.m 


1 • 


||(l-A(i))z||*  - (,‘'||(s+»I)-’rU'j||2 

•t  m 


We  now  discuss  the  determination  of  U.  It  can  be  seen  that  U 
Is  any  matrix  whose  n-M  columns  are  orthonortnal  and  perpendicular  to  the  M 
columns  of  T,  T ■ We  obtained  U as  follows.  Let 


I - HT'T)"'  T'  - U A U'  . 


where  U Is  orthogonal  and  a Is  diagonal.  Since  I-T(T'Ti “H' Is  a projection 
matrix  of  rank  n-M,  a Is  a matrix  with  M zeroes  and  n-M  ones  on  the  diagonal. 
We  used  EISPACK  (Smith  et.  a1.  (1976))  to  perform  the  eigenvalue  decomposition 
U A U'  and  the  n-M  columns  of  U are  taken  as  the  column:;*  of  0 corresponding 


to  the  n-M  ones  In  a.  Each  such  vector  Is  pcrpendicula  to  the  columns  of  T, 


IT 
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as  can  be  seen  by  right  multiplying  (18)  by  T.  The  EISPACK  computation  of  the 
entries  of  l was  good  to  seven  figures.  Given  U,  B Is  computed  and  r and  6 are 
also  computed  using  the  eigenvalue  decomposition  routines  In  EISPACK. 


5.  Numerical  Results 


We  present  the  results  of  a single  Monte  Carlo  experiment,  with  m>2.. 
Figure  1 gives  a picture  of  the  true  function  u that  was  the  subject  of  the 
first  experiment, 

-^({x-2)V) ^ — 7((x+2)V)' 


u(x,y) 


1 


2tt(1.3)‘ 


2(1. 3)‘ 


+ e 


2(1. 3)‘ 


A regular  7x7  square  array  of  49  points  t^ . 1 - 1,2.... .49  was  selected,  with 
the  middle  point  being  ( 0,0  ) and  the  point  spacing  being  1.0.  Data  y^ 
were  generated  as 


y^  ■ u(t^)  + , t^  ■ (x^.y^)  . 1-1,2 49 

2 

where  the  were  pseudorandom  N(0,o  ) random  variables  with  o ■ .01.  a Is 
about  1/8  of  the  maximum  height  of  u.  Figure  2 presents  a picture  of  the 
data  points,  which  have  been  Joined  by  straight  lines.  Figures  3 and  4 give 
'^n,2,x  values  of  x.  In  Figure  3.  \ Is  too  large,  and  In  Figure  4,  \ Is 

too  small.  Figure  5 gives  u.  9 where  X Is  the  mlnlmizer  of  V(x).  Figure  6 

T\fC  *A 

gives  a plot  of  R(x)  and  V{x}  against  log  X.  It  Is  seen  that,  In  the 
neighborhood  of  the  mlnlmizer  of  R(x),  V(x)  roughly  follows  R(x).  Theoretically, 


we  have  min  EV(x)  2^  min  ER(x)  0^,  for  large  n.  see  CW.  and  this  relationship 
X X 

Is  roughly  approximated  here.  The  achieved  Inefficiency,  defined  by 


R(x)/m1n  R(X),  where  x Is  the  mlnlmizer  of  V(x},  was  1.54.  Note  that 
X 2 

min  R(x)  ■ .25  0 . If  we  were  fitting  a surface  which  Is  known  to  be  a linear 


combination  of  given  functions  by  regression  we  would  expect  the  mean  square  error 

.2 


to  be  proportional  to  — . Here  numerical  and  theoretical  results  In  the  one 
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ZflXIS 


Figure  3 


Figure  5 


R(X).  V(x),xio 


dlmenslonnl  case  for  reasonably  regular  arrangements  of  data  points  indicate 

that  min  R(x)  const.  (o^/n)P  where  p is  some  power  slightly  less  than  one.  p 
X 

depends  on  the  rate  of  decay  of  eigenvalues  of  an  appropriate  reproducing 
kernel.  See  Wahba  and  Wold  (1975).  CW.  and  Wahba  (1975b,  1977).  If 
u c H^"’(R^),  p ■ 2m/(2m+l).  (In  preparation). 

Nr.  Wendelberger's  program  is  running  for  n ■ 120  and  quite  reasonable 
results  have  been  obtained  for  this  case,  with  randomly  chosen  points  {t^}. 

One  cannot  Increase  n with  impunity,  however.  In  the  n ■ 49  case  reported 
here  the  condition  number  of  B,  namely  max  b^/m1n  b^  was  around  200,  and  in  the 
irregularly  spaced  n ••  120  case  this  condition  number  was  of  the  order  of 
4x10^.  (Irregularly  spaced  points  will  Increase  the  condition  number.) 

For  large  n and  a condition  number  somewhere  around  (we  guess)  10^  or  10^, 
the  computation  errors  will  begin  to  take  over.  Thus,  in  theory,  a plot  of 
1og(m1n  R(x))  vs.  n should  be  approximately  linear  with  slope  -p,  however,  as 
roundoff  error  gets  large,  this  plot  will  flatten  out.  Laurent  and  colleagues 
(1978)  have  developed  a procedure  for  patching  together  surfaces  of  this  type 
so  that  groups  of  points  may  be  handled  separately. 

The  cost  of  running  a program  designed  Just  to  produce  Figure  5 from 
data,  we  estimate  to  be  about  $4.00  at  weekend  rates  at  our  computing  center. 

To  produce  Figure  5 from  a second  set  of  data  at  the  same  points  (t^l,  one 
would  retain  U,  r and  (b^),  which  depend  on  the  (t^l  but  not  z,  and  then  the 
cost  would  be  very  small. 


6.  Niscellaneous  Remarks 

We  hope  to  Implement  the  m ■ 3 and  m - 4 cases,  n can  then  be  selected 
from  the  data  by  comparing  V(x)  for  each  of  the  m ■ 2,  3 and  4 cases.  For 
m ■ 2,  the  roughness  penalty 
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Is  the  bending  energy  of  a thin  plate.  For  this  reason.  Duchon  christened  the 
solutions  "plaque  mince"  splines.  We  have  reason  to  believe  that  the  m - 3 case 
will  be  appropriate  for  the  smoothing  of  certain  meteorological  data.  In  some 
cases  the  nature  of  the  physical  phenomena  being  smoothed  may  provide  Insight 
Into  a choice  of  m. 

We  note  that  the  solutions 

“n.m.xit  " ° ^ ^ • 

2 2 

where  A Is  the  Laplaclan  operator  au  ■ ^ . The  smoothing  splines 

iom\ 

^n.m.x  ^n.m.x^^^  ■ 0,  t f t^,...,tj^.  For  this  reason,  Prof.  Iso 

Schoenberg  has  suggested  to  us  that  the  functions  be  called  "Laplaclan 

Smoothing  Splines". 

We  have  recently  obtained  what  might  be  called  the  Laplaclan  hlstospllnes, 

by  analogy  with  Boneva.  Kendall,  and  Stefanov  (1971).  These  are  functions 

which  minimize  the  roughness  penalty  I //  (7)  ( — r m r)  subject  to 

J-0  A ^ Sx'^ay 

volume  matching  conditions  of  the  form 

u dxdy  ■ p^  , 1 • 1,2 n 

where  the  are  n bounded  areas  In  r"  whose  union  Is  A.  These  functions  satisfy 

A^u  ■ constant  on  each  A^  . 

(Qyn,  Wong,  and  Wahba  (1979)),  and.  In  preparation). 

Various  optimality  properties  of  smoothing  splines  and  hlstospllnes  In  one 
dimension  are  known.  For  example.  It  can  be  shown  from  CW  and  Wahba  (1975b)  that 

E j|(fn,„,x(t)-f(t))^dt  - o(n“^^'">/^^"''^^J).  f c h'"(0.1) 

. f e h2"’(0.1) 
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and  f satisfies  some  ^ ^undary  conditions.  It  is  part  of  the  folklore  that 


these  rates  cannot  be  improved  upon.  Density  estimates  determined 
by  the  mlnimizer  of  ^(f^"'^(t))^dt  subject  to  the  area-matching  conditions 

*1+1 

/ f(t)dt  ■ fraction  of  observations  In  [t^.t^^.^] 

*1 

are  known  to  achieve  the  best  possible  convergence  rates  over  f c h"*  provided 
the  t^  are  chosen  properly.  See  Wahba  (1975c,  1976).  Stone  (1978)  has  given 
some  results  on  best  possible  pointwise  convergence  rates  in  d dimensions. 

We  conjecture  that  all  the  nice  convergence  properties  of  polynomial  splines 
can  be  extended  to  the  Laplacian  smoothing  splines  and  Laplacian  histosplines. 


APPENDIX 


Outline  of  Proof  of  Theorem 


Let  be  a subset  of  M points  selected  from  t^i...,t^ 

with  the  property  that  the  MkM  matrix  T with  entry  Is  of  full 

m V J 


3% 


rank.  The  space  H ■ {u:  u e t»',  — r-'  e Ut  J ■ 0,l,...,m-l}  can  be 

gjjljyin-J 

decomposed  Into  the  direct  sum  of  two  spaces: 


H “ 1 , 

m- 1 *“ 

where  rr^^i  is  the  M dimensional  space  of  polynomials  of  total  degree  m-1 
or  less  and  7 » (u:  u e H,  u(r^)  ■ 0,  v ■ 1 ,2,... ,M).  It  can  then  be  shown 
that 


m M «m,,  .m,, 

<u,v5y  h //  I ( j)  — T~-aT 

- j*0  ^ ax^ay"*^  3xW^ 


defines  an  Inner  product  on  7.  If  an  Inner  product  Is  defined  on  , by 
M - 

<u,v>  ■ ][  u(r^)v(r^),  then  and  7 a*^®  orthogonal  subspaces.  7 (and 

m-1  V“1 

and  hence  H)  are  reproducing  kernel  spaces. 

If  the  reproducing  kernel  K(S|t)  for  7 can  be  found,  then  the  solution 
^n,m,x  minimization  problem  of  (4)  will  have  a representation 


n - 


M . 


■ jJi  ‘j  '^v  ♦vtt)  • 

(See,  e.g.  Kimeldorf  and  Wahba  (1971)}.  u „ will,  of  course,  be 

n,m,  A 

Independent  of  the  choice  of  ri|,...,r|^.  The  reproducing  kernel  K has  been 
found  by  Meinguet  (1978,  1979)  and  Is  given  by 


(A.l) 
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H 


M 

> 1 
V"1 

Pv<^) 

M 

+ I 

M.V 

p„(0  P,(t) 

(A.Z) 


Where  span  'iru|_i  and  are  chosen  so  that  Py(r^)  ■ 1»  w ■ w.  ■ 0,  vi  f v. 

Substituting  (A.Z)  Into  (A.1),  It  Is  seen  that  a representation  of  the  form 
(5)  for  u»  _ . holds. 

niHiiA 

To  show  that  K Is  the  reproducing  kernel  for  Xi  It  Is  necessary  to  show 


that 


I ) K(Si')  c 7 . each  s 

II)  <K(s,.).  K(t, •)>){■-  K{s,t)  , 


where 


Define 


Then 


M m a"*.! 

■ /ni\  3 u 


H,(t)  - EJs.t)  . I pjs)  E„(r,.,t)  . 


(A.3) 

(A.4) 


w“l 


w'  ' m'  u' 


K(s.t)  - H.(t)  - I p^(t)  H.(r^)  . 


s'  ' ‘‘i  ' s'  v' 


(A.5) 


The  hard  part  Is  to  show  that  e H.  (Note  that  E^^  ^ H.)  Meinguet  shows 

that  H-  e H,  for  each  s,  and  we  omit  the  proof.  It  then  follows  that 
* M 

K(s,*)  c W,  and,  since  I p (•)  H (r  ) Is  the  polynomial  Interpolating  to 


V"1 


Hj  at  r^,...,rn^,  K(s,r^)  - 0,  v - 0,1 M,  and  so  K(s,*)  e 7 . 

To  establish  (A.3),  first  note  that 

".<•>  • 


(A.6) 
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Consider  the  Green's  formula 


3'"u 


J«0  8x  sy'" 


— j dxdy  - //  dxdy 
ax  ay  •' 


where  a 


This  formula  holds  provided,  e.g.  v e H (1  L. 


If  u e P,  then  the  potential  formula 


//(A'"u)(t)  E^(s,t)dt  • u(sj 


holds  (see  Schwartz  (1966))  and  In  particular 
//a"*  u • Hj  - u(s)  - ? Pjs)  u(r  ) . 

V 

Meinguet  argues  that,  In  fact  (A. 7)  and  (A. 8)  hold  for  u ■ H^,  v ■ 
— m _ .m  .m 

(■1)  I (7)  //  =-T  H.  • H- 

J"0  a>ray^  ^ airay^  * 


■ H^(s)  - p^(s)  H5(r^)  5 K(s,t)  , 

which,  combined  with  (A. 6),  gives  (A. 3). 

Equation  (7)  can  be  obtained  as  follows:  Considering  K(t,tj) 

of  t, 

K(t.tj)  . E„U.t^)  - E„t,.r,)  t 

a polynomial  of  degree  m-1  or  less. 

Now,  1f  ^ Is  any  element  of  we  have 
M 

♦(t)  - I Py(t)  ♦(r  ) s 0 . 

v-1  ''  '' 

Letting  a^(J),02(J),...,a^(d)  be  the  coefficients  of  Ej|^(*,t^),  E|j|( 
...,  Euj(«,t^),  In  (A. 9),  It  can  be  verified  from  (A.IO)  that 


(A.7) 

and  u e P. 


(A.8) 

Hj,  giving 


as  a function 

(A.9) 

(A.IO) 

•^2) , 


n 

I °*L ( J ) = Oi  J ® t 

k«l  ^ 

which  results  directly  in  the  conditions  (7)  on  the  coefficient  vector  c in 
(5),  namely,  Tc  ■ 0.  Equation  (6)  is  obtained  as  follows:  One  substitutes 

Nt 

(A.l)  into  (4),  and  then  uses  (A. 3)  to  evaluate  the  expression  (4)  to  be 
minimized.  By  repeatedly  using  T'c  - 0,  one  obtains  that  c and  d are  chosen 
! subject  to  T'c  > 0,  to  minimize 

||z-Kc-Td||^  + n X c'  K c . 

**  M M M 

f, 

f Differentiating  this  expression  with  respect  to  c and  setting  the  result  equal 

; to  zero,  and  using  T'c  ■ 0,  gives  (6). 

i 


i 

i*. 


f. 


r 

i 
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ACRONYMS  AND  SYMBOLS 


APDS 


nm 

OC 

PE 


Armor  Piercing  Discarding  Sabot 
Millimeter 

Operating  Characteristics 
Probable  Error 


CPE. 
• c 

CPE. 

CPEg 

CPEj 

CPE, 

K 

X 

n 

N 


Characteristic  Circular  Probable  Error 

Estimate  of  Characteristic  Circular  Probable  Error 

Level  of  Circular  Probable  Error  which  represents 
good  lot  quality 

Inherent  Circular  Probable  Error 
Circular  Probable  Error  which  characterizes  a lot 
Observed  Circular  Probable  Error 
A Constant 

Ratio  of  CPEjj  to  CPEj. 

Sample  Size 
Number  of  occasions 


PE 


“J.  "R- 

*9 

oj,  oj,  oj,  etc. 
S^ 

s?.  sj.  s|.  .tc. 


Estimate  of  Probable  Error 
Population  Variance 

Population  Variance  of  a set  of  x,  y and  radial 
observations,  etc. 

2 

Estimate  of  o** 

2 2 2 

Estimates  of  o‘,  oj^,  etc. 

Calculated  variance  of  a sample 

Calculated  variance  of  a set  of  x,  y and  radial 
observations,  etc. 
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METHODOLOGY  FOR  ACCEPTANCE  CRITERIA  FOR 
TARGET  DISPERSION  CHARACTERISTICS  OF  THE 
ARMOR  PIERCING  DISCARDING  SABOT  (APDS)  ROUNDS 

1 . INTRODUCTION 

1.1  General 


The  estimation  of  target  dispersion  characteristics  of  Armor 
Piercing  Discarding  Sabot  (APDS)  ammunition  In  acceptance  testing  Is 
accompanied  by  a number  of  difficulties.  Test  costs  are  high;  therefore 
sample  sizes  are  limited.  Since  dispersion  patterns  are  subjected  to 
relatively  large  variations!  small  sample  sizes  produce  undesirable 
levels  of  Inaccuracy  In  estimating  dispersion  characteristics.  Control 
of  test  conditions  throughout  the  test  Is  limited  to  control  of  only  a 
few  factors  such  as  tube  aim  point,  cant,  stability  of  the  firing  plat- 
form and  tube  condition,  at  the  start  of  the  test.  Even  with  maximum 
possible  control  of  such  factors,  they  still  exert  a degree  of  error  In 
round-to-round  target  Impact  points.  In  addition,  there  are  several 
uncontrollable  factors,  such  as  wind  and  weather  conditions,  tube  wear 
from  round  to  round,  droop.  Jump  and  other  unknowns,  which  make  It 
impossible  to  obtain  uniform  conditions  throughout  the  test.  These 
problems  have  accompanied  every  acceptance  test  conducted  on  APDS 
ammunition  and  have  been  exacerbated  by  two  factors: 

0 The  lack  of  established  test  procedures  designed  to 
minimize  the  effects  of  uncontrollable  test  condition 
variations  and  t- 

0 The  lack  of  established  acceptance  criteria  and 
estimation  procedures  designed  to  minimize  consumer 
and  producer  risks. 

The  result  has  been  that  a large  number  of  lots  of  APDS 
aimunltlon  with  very  poor  target  dispersion  characteristics  have  been 
accepted  for  use. 


<1 


il 


1.2  Purpose  | 

ii 

This  report  develops  methodology  which  can  be  used  to  derive  I 

acceptance  plans  for  target  dispersion  characteristics  of  APDS  rounds.  | 

In  developing  the  methodology,  the  effect  of  test  condition  variations  | 

upon  target  dispersion  patterns  and  the  lack  of  established  acceptance  | 

criteria  and  estimation  procedures  are  addressed.  Examples  of  Inadeouate  | 

firing  procedures  In  accuracy  tests  of  APDS  rounds  are  presented,  and  j 

corrective  measures  A*^e  proposed.  Examples  of  acceptance  criteria  and 
estimation  procedures  which  minimize  consumer's  and  producer's  risks  are  i 

developed..  Several  acceptance  plans,  derived  from  the  proposed  method- 
ology are  presented. 
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2.  TEST  PROCEDURES 
2.1  Background 


The  objective  of  an  accuracy  acceptance  test  of  APDS  rounds  Is 
to  assess  the  dispersion  pattern  which  characterizes  a lot  of  ammunition 
and  accept  or  reject  the  lot.  The  acceptance  test  requires  firing  a 
group  of  rounds  at  a vertical  target  some  distance  from  the  gun.  The 
coordinates  of  the  Impact  points  of  each  round  In  the  group  are  obtained, 
and  estimates  of  the  dispersion  about  the  center  of  Impact  are  determined. 

When  firing  a group  of  rounds  to  assess  dispersion  character- 
istics, It  Is  desirable  to  nave  Identical  test  conditions  for  each  round. 

In  this  manner,  the  differences  In  Impact  points  of  each  round  are  the 

result  only  of  Inherent  differences  between  the  rounds.  Inherent 
differences  between  rounds  In  a lot  of  APDS  aimunltlon  are  due  to  chance 
variation  within  a stable  pattern  caused  by  manufacturing  procedures  and 
physical  characteristics  of  the  round  and  propelling  charge.  These 
Inherent  differences  lead  to  different  flight  characteristics,  and  cause 
rounds  to  Impact  at  different  points  on  the  target.  If  Identical  test 
conditions  are  obtained,  the  dispersion  characteristics  of  the  group  of 
rounds  fired  reflect  the  degree  of  round-to-round  uniformity  In  the 
manufacturing  process  and  provide  an  estimate  of  quality  control. 

Unfortunately,  test  conditions  from  round  to  round  are  not 
Identical.  Gun  elevation  and  deflection  vary  regardless  of  efforts  to 
maintain  a constant  aim  point.  Weather  conditions  and  other  factors 
which  effect  accuracy  also  vary  from  round  to  round.  Consequently, 
the  dispersion  pattern  of  a group  of  rounds  on  a target  Is  not 
representative  of  the  Inherent  round-to-round  differences.  The 
dispersion  pattern  consequently  represents  the  combination  of  the 
Inherent  differences  In  rounds  and  the  variability  In  test  conditions 
from  round  to  round. 

2.2  Firing  Procedures 

The  method  of  firing  employed  In  an  acceptance  test  of  a lot  of 
APDS  ammunition  must  be  conducted  In  a way  that  minimizes  the  effect  of 
round-to-round  variability  In  test  conditions.  In  past  acceptance 
tests,  gun  elevation  and  deflection  settings  have  been  controlled 
to  a great  extent,  and  severe  weather  conditions  have  been  avoided. 
However,  the  method  of  firing  In  acceptance  tests  has  not  been  one  which 
minimized  round-to-round  variability  In  test  conditions.  The  length 
of  time  required  to  fire  a group  of  rounds  has  been  as  great  as  four 
hours.  Test  conditions  such  as  tubs  droop,  cant,  ambient  environmental 
conditions  and  other  unknowns  vary  more  over  a Iona  time  Interval  than 
they  do  in  a short  one.  Hence,  as  shown  by  an  analysis  of  past  acceptance 
tests  of  lOSiTin,  APDS  anmunltlon,  a group  of  rounds  fired  over  a long 
time  period  will  tend  to  exhibit  higher  probable  errors  than  would  be 
observed  over  a short  time  period. 
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Figure  1 shows  the  accuracy  results  of  an  acceptance  test  of 
105nin,  APDS  rounds  conducted  at  Jefferson  Proving  Ground.  Horizontal 
and  vertical  probable  errors  are  presented  as  a function  of  time  between 
rounds.  The  wind  ranged  from  2-10  knots  and  varied  In  direction  from 
140*  to  180*  during  the  course  of  the  test.  The  probable  errors  for  the 
entire  25  round  group  were  calculated  at  0.47  mils  In  the  horizontal 
direction  and  at  0.29  mils  In  the  vertical  direction.  Probable  errors 
as  a function  of  time  between  rounds  were  obtained  by  analyzing  all 
combinations  of  two  round  groups  In  the  test.  The  probable  errors  for 
each  two  round  group  were  calculated  and  correlated  with  time  between 
rounds  fired.  Although  the  trend  In  Figure  1 Is  linear,  other  shapes 
may  be  expected  from  the  testing  of  other  lots. 

Figure  1 clearly  Illustrates  that  during  an  accuracy  test  the 
dispersion  of  Impacts  Is  greatly  affected  by  the  test  conditions,  which. 
In  turn,  vary  with  time. 

The  firing  procedures  employed  In  acceptance  tests  have  not 
been  designed  to  minimize  the  time  over  which  a group  of  rounds  Is  to  be 
fired.  In  the  past,  as  many  as  three  different  lots  were  often  tested 
simultaneously,  with  rounds  from  each  lot  fired  alternately,  with 
reference  rounds.  The  effect  quadrupled  the  amount  of  time  required 
to  fire  each  test  group  of  each  lot.  Consequently,  the  effect  on 
dispersion  due  to  variations  In  round-to-round  test  conditions,  has 
been  greater  than  that  which  could  have  been  obtained  If  the  time  for 
firing  each  group  were  reduced. 

In  conducting  an  acceptance  test,  the  Individual  groups  of 
rounds  from  a test  lot  should  be  fired  sequentially  with  no  alternate 
firing  of  reference  rounds  or  rounds  from  other  test  lots  between 
rounds  within  a group.  The  time  for  firing  each  group  of  rounds 
can  thus  be  minimized  to  the  greatest  extent  possible.  If  reference 
rounds  are  to  be  fired,  each  group  of  reference  rounds  should  be  fired 
either  before  or  after  each  group  of  test  rounds.  For  example,  If  two 
ten  round  samples  from  a single  lot  are  to  be  tested  with  fifteen 
reference  rounds,  the  order  of  firing  could  be: 

Five  Reference  Rounds 

Ten  Sample  Rounds 

Five  Reference  Rounds 

Ten  Sample  Rounds 

Five  Reference  Rounds 

Estimates  of  probable  errors  for  each  group  of  rounds  would 
then  be  calculated  and  pooled  accordingly. 
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3.  CALCULATION  PROCEDURES 


When  a group  of  rounds  1s  fired  over  a short  time  Interval,  the 
effect  of  nonuni form  test  conditions  from  round  to  round  will  still 
persist.  Movement  of  the  mean  center  of  Impact  from  round  to  round  may 
occur,  and,  If  so,  the  effect  of  this  trend  on  calculated  probable  errors 
may  be  eliminated  by  the  method  of  successive  differences  (Reference  1). 
For  example,  suppose  the  following  Impact  coordinates,  measured  In  Inches, 
are  obtained  for  a group  of  ten  rounds  fired  at  a vertical  target  1000 


maters  from  the  gun: 

Round 

Horizontal 

Vertical 

Number 

Coordinate  (x) 

Coordinate  (vl 

1 

65 

no 

2 

70 

120 

3 

60 

115 

4 

75 

100 

5 

70 

105 

6 

85 

95 

7 

80 

90 

8 

95 

95 

9 

90 

85 

10 

100 

75 

Calculating  probable  errors  In  the  usual  manner  for  the  entire 
ten-round  group  (References  2 and  3)  yields. 

Horizontal  probable  error  ■■  0.23  mils 
Vertical  probable  error  ■ 0.24  mils 
Circular  probable  error  ■ 0.41  mils. 

Calculating  probable  errors  using  the  method  of  successive 
differences  yields. 

Horizontal  probable  error  ■ 0.13  mils 
Vertical  probable  error  > 0.11  mils 
Circular  probable  error  <•  0.21  mils. 

The  method  of  successive  differences  results  In  approximately 
a SO  percent  reduction  In  probable  error  estimates  In  this  particular 
example.  The  probable  errors  calculated  by  the  standard  method  include 
the  effects  of  test  condition  variability  over  the  entire  ten-round 
group,  while  the  probable  errors  calculated  by  the  method  of  successive 
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differences  Include  only  the  test  condition  variability  between  successive 
rounds.  Whenever  the  variation  In  test  conditions  Is  nonrandom,  the 
result  Is  a nonrandom  dispersion  pattern  of  shots  about  the  center  of 
Impact.  A trend  of  this  type  Indicates  that  test  condition  variability 
over  the  entire  group  Is  greater  than  the  variability  between  successive 
rounds.  When  this  type  of  trend  occurs,  the  use  of  the  method  of  success- 
ive differences  provides  dispersion  estimates  which  Include  only  the  effects 
of  round-to-round  test  condition  variability.  In  this  manner,  the  effect 
of  variability  over  the  entire  group  Is  eliminated. 

In  determining  the  dispersion  characteristics  of  a lot  from  a 
group  of  rounds,  It  Is  therefore  desirable  to  limit  the  effect  of  test 
condition  variability  on  the  estimates  of  dispersion  to  that  variability 
which  occurs  only  between  rounds. 

It  can  be  shown  that  the  method  of  successive  differences 
provides  an  unbiased  estimate  of  the  square  of  the  probable  error  (PE). 

From  statistical  theory,  an  unbiased  estimate  of  the  variance  (o^)  In 
the  X direction  of  a two-round  sample  Is  obtained  by  ^ 

(x-j  - Xg)^,  where 

x-j  and  X2  are  the  coordinates  of  Impact  on  the  x axis.  If  three  rounds 
are  fired  and  have  coordinates  of  Impact  x^  X2  and  X3  on  the  x axis, 

hi  ■ + *"'*  ®2x  ■ 4-  (»2  - *3)* 

provide  two  unbiased  estimates  of  the  variance,  cr^^.  The  sample  variance 
calculated  by  the  method  of  successive  differences  Is  ■ 1/2 
1/2  $2^,  and  It  Is  an  unbiased  estimate  of  0^.  Since 

£($2)  " E(l/2  Syl  + 1/2  $2^) 

- E(l/2  S^l)  + E(l/2  $2^) 

- 1/2  E(S,2)  y2 


■ o\/Z  + oJ/2 
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wh«i*«  hi  • KsJ  and  E(sJ)  ■ oj. 

H«nc«i  the  mthod  of  succassivo  differances  givas  an  unblasad  astimata 
of  tha  squara  of  tha  probabla  error  between  rounds  In  the  x direction. 

A similar  proof  shows  that  tha  mathod  of  successive  differences 
also  provides  an  unbiased  estimate  of  the  probabla  error  squared  between 
rounds  In  the  y direction  and  also  In  the  radial  direction  (circular 
probable  error). 

Consequently,  the  use  of  the  method  of  successive  differences 
provides  unbiased  estimates  of  the  probable  error  squared  and  eliminates 
the  effect  of  nonrandom  test  condition  variation  upon  the  dispersion 
results  of  an  entire  group  of  rounds. 

Whenever  the  variation  In  test  conditions  Is  nonrandom  and. 
thus,  results  In  a dispersion  pattern  which  Is  not  random  about  the 
center  of  Impact,  the  method  of  successive  differences  should  be  employed. 
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4.  ACCEPTANCE  CRITERIA 


4.1  Applicable  Parameters 

In  the  development  of  acceptance  criteria  for  dispersion  character 
Istics  of  APDS  rounds,  parameters  appropriate  to  an  accept/reject  decision 
must  be  selected.  Horizontal  and  vertical  probable  errors  could  be  the 
basis  for  a decision  with  Independent  criteria  for  each.  This  has  the 
disadvantage,  however,  of  not  utilizing  all  available  Information.  For 
example,  the  observed  horizontal  and  vertical  probable  errors  In  a test 
could  be  0.10  mils  and  0.3S  mils,  respectively.  The  pooled  average  of 
these  is  0.26  mils.  If  the  reject  criterion  Is  to  reject  lots  when  either 
horizontal  or  vertical  probable  error  Is  greater  than  0.30  mils,  this  lot 
would  be  rejected.  The  problem  with  this  type  of  accept/reject  criteria 
Is  that  It  Ignores  good  dispersion  characteristics  In  one  direction  when 
dispersion  1n  the  other  direction  Is  poor. 

Results  of  acceptance  tests  end  life  cycle  evaluations  of  105mm, 
M392,  APOS  rounds  have  shown  that  target  dispersion  patterns  are  approx- 
imately circular.  In  Reference  4,  for  example,  horizontal  and  vertical 
probable  errors  were  0.19  m11s  and  0.21  mils,  respectively  for  803 
rounds  fired  from  a mid-life  tube.  Hence,  the  use  of  circular  probable 
error,  which  effectively  combines  all  dispersion  Information  In  both  the 
horizontal  and  vertical  directions.  Is • appropriate.  Use  of  this  parameter 
simplifies  the  aceept/rejeet  criteria.  Another  advantage  of  using  the 
circular  probable  error  In  estimating  target  dispersion  rather  than  the 
present  technique  of  computing  Independent  horizontal  and  vertical 
probable  errors.  Is  that  the  sample  size  requirement  Is  reduced 
significantly  for  the  specified  risk. 

4.2  Distribution  Of  Circular  Probable  Errors 

A lot  has  an  Inherent  circular  probable  error,  CPEj,  which 

describes  Its  expected  performance  when  random  samples  from  the  lot  are 
fired  under  Identical  test  conditions.  If  CPEq  Is  the  circular  probable 

error  observed  for  a random  sample  fired  under  Identical  test  conditions, 
then 

E(CPEq)  - CPEj 

Since  circular  probable  error  Is  a multiple  of  the  radial 
standard  deviation  It  follows  that. 
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whers  S|^  Is  the  observed  radial  standard  deviation,  and  Is  the 

expected  radial  standard  deviation  for  a random  sample  fired  under 
Identical  conditions. 


2(n-l)CPE? 

The  distribution  of therefore  be  Chi-square, 

CPE| 

with  2(n-1)  degrees  of  freedom  for  a random  sample  of  size  n fired  under 
Identical  test  conditions. 

Identical  conditions  from  round  to  round  are  not  attainable 

Cpg2 

during  testing,  however.  Therefore.  -^w111  not  have  a Chi-square 

CPEJ 

distribution  during  tests.  In  fact,  CPEj  cannot  be  adequately  estimated 

from  test  results,  since  the  effects  of  round-to-round  variability  In 
test  conditions  will  always  be  Included  In  circular  probable  error 
estimates. 

To  develop  acceptance  criteria  for  APDS  rounds,  It  is  necessary 
to  know  the  form  of  the  distribution  of  CPEq.  It  Is  also  necessary  to 

estimate  a circular  probable  error,  characterizing  a lot,  which  can  be 
determined  from  test  data. 

During  acceptance  testing,  the  effect  of  variation  In  test 
conditions  from  round  to  round  on  circular  probable  error  estimates  will 
vary  from  one  occasion  to  another.  On  some  days,  test  condition 
variability  has  little  effect  on  dispersion  patterns,  while  on  other  days, 
the  effect  of  variable  test  conditions  Is  comparatively  large.  Obvlousiyi 
measurements  which  characterize  a lot  or  Its  expected  dispersion  should 
not  be  based  on  days  when  test  condition  variability  Is  unusually  small 
or  large.  The  measurement  which  adequately  characterizes  the  performance 
of  a lot  should  be  based  on  the  outcome  expected  on  a randomly  selected 
day.  given  that  the  day  satisfies  the  meteorological  requirement  for 
conducting  an  acceptance  test. 

To  define  characteristic  circular  probable  error,  I.e.,  the 
probable  error  which  Is  expected  from  a lot  on  a random  day,  we  assume 
that  a lot  of  Infinite  size  Is  available.  Let  N random  samples  be 
selected  from  the  lot  and  let  each  sample  be  tested  on  a random  day. 

Let  CPE,,  denote  the  characteristic  circular  probable  error  of  the  lot. 
Then, 


1 

\ 
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defines  the  characteristic  circular  probable  error  of  the  lot.  In  the 
above  equation,  (CPEg)^  1s  the  square  of  the  circular  probable  error 

observed  on  the  l^*’  occasion.  Since  the 

expected  value  of  the  observed  circular  probable  error  cn  a randomly 
selected  day,  It  Is  clear  that  CPE.  adequately  defines  the  performance 
of  the  lot.  ^ 

On  some  days,  E(CPEg)  f CPEg,  since  variability  In  test  conditions 

such  as  wind  and  weather,  tube  wear  from  round  to  round,  droop.  Jump, 
etc.,  may  be  unusually  large  or  small.  Suppose  conditions  on  a given 
day  are  such  that  E(CPEg)  ■ CPEg  for  the  random  samples  fired  on  that 

day.  Call  this  an  average  day. 

Now.  looking  only  at  tests  conducted  on  average  days  (days  for 
which  E(CPEq)  ■ CPEq),  there  Is  a characteristic  radlai  standard 

deviation,  Oq,  which  describes  the  dispersion  characteristics  of  the  lot. 

Since  circular  probable  error  Is  a multiple  of  radial  standard  deviation, 
then. 


where  CPEq  Is  the  circular  probable  error  observed  for  a random  sample 
tested  on  an  average  day,  and  Is  the  estimate  of  the  variance.  The 


distribution  of 


2(n-l)  CPE 

“Iref" 


2 

2,  will  therefore  be  Chi-square  with 


degrees  of  freedom  for  a random  sample  of  size  n tested  on  an 
day. 


2(n-l) 

average 


Acceptance  tests,  however,  are  not  usually  conducted  an  average 

days,  and 


has  In  actuality  greater  variability  than  that  predicted  by  the 
Chi-square  distribution.  To  assess  how  this  ratio  varies  during  acceptance 
tests,  the  dispersion  results  of  176  groups  of  lOSinm,  APDS,  M392  reference 
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rounds  were  analyzed.  Eight  of  these  groups  had  target  misses  and 
were  excluded  from  further  analysis.  The  remaining  groups  were  tested 
on  168  different  days  over  a twelve  year  period.  Each  group  consisted 
of  a ten  round  sample  from  one  of  two  reference  lots,  and  was  fired  at 
a target  2000  meters  from  the  gun.  There  was  no  significant  difference 
In  the  distribution  of  circular  probable  errors  of  each  reference  lot 
and  results  from  the  two  lots  were  therefore  combined.  The  estimate 
of  the  characteristic  circular  probable  error  of  the  reference  lotSi 
CPEqi  was  obtained  from  the  following  equation. 


CPEc- 


TW  ,1, 


ml) 


OM 


1/2 


where  (CPEg)^  was  the  square  of  the  circular  probable  error  observed 
on  the  1^^  occasion. 


CPEq 

In  order  to  facilitate  the  analysis,  -^was  defined  to  be  X, 

CPEg. 


Assuming  that  CPEg  for  the  168  reference  groups  was  equal  to  the 
characteristic  circular  probable  error  of  the  reference  lot.  then 


CPE, 


11 


CPE, 


For  each  of  the  168  groups,.  was  determined  and  the  observed  cumulative 
distribution  of  the.  x^'s  was  plotted.  Figure  2 presents  the  observed 
cumulative  distribution  of  X.  and  compares  It  to  the  cumulative 

* O 

distribution  which  would  result  If  x^  were  distributed  as  a Chi-square 
with  18  degrees  of  freedom. 


From  Figure  2.  It  Is  evident  that  If  2(n-l}X^  were  distributed 

as  a Chi-square  distribution  with  18  degrees  of  freedom.  80  percent  of 
the  observations  would  be  between  0.78  and  1.20.  The  observed  cumulative 
distribution  of  the  X|'t.  however,  shows  that  80  percent  of  the 

observations  were  between  0.46  and  1.44,  a considerably  wider  spread  than 
that  predicted  by  the  Chi-square. 


The  Gamma  distribution  of  the  form  Ganna  (X) 
was  fitted  to  the  observed,  x^'s.  With  a ■ 7.4558  and  0 


j^B-1 

(o-l)l0® 
0.1261,  the 


X 

? 


Gamma  distribution  fits  the  data  very  well.  The  Chi-square,  Cramar-Von 
Nises  and  Kolmogorov-Smimov  goodness  of  fit  tests  gave  no  reason  to 
reject  the  Gamma  distribution.  A suimary  of  the  observed  x.|'s  and  of 

the  fitted  Gamma  distribution  are  presented  1n  Figure  3.  Each  data 
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point,  denoted  by  a triangle,  represents  the  number  of  observations 
within  an  interval  of  length  0.10.  Points  from  the  Ganna  distribution 
were  multiplied  by  16.8  so  that  the  data  and  fitted  curve  could  be  shown 
on  the  same  scale.  In  Figure  4,  the  cumulative  Gamma  distribution  is 
compared  with  the  observed  cumulative  distribution  of  the  x^'s.  The 

data  points,  denoted  by  triangles,  represent  the  observed  cumulative 
probability  that  x^  <,  x. 

4.3  Criteria  For  Acceptance  Plans 

Assuming  that  x follows  the  Gamma  distribution  with  a ■ 7.4558 
and  6 ■ 0.1261  for  random  samples  of  size  10,  acceptance  plans  can  be 
derived  with  various  levels  of  consumer's  risks  (probability  of  accepting 
a lot  with  poor  quality)  and  producer's  risks  (probability  of  rejecting 
a lot  with  good  quality).  The  limitation  of  using  this  Gamma  distribution 
is  that  it  can  only  be  used  to  calculate  consumer's  and  producer's  risks 
for  acceptance  plans  with  sample  sizes  which  are  multiples  of  ten. 
Derivations  of  distributions  applicable  to  sample  sizes  other  then  multi- 
ples of  ten  are  beyond  the  scope  of  this  report.  To  develop  an  acceptance 
plan,  it  is  necessary  to  specify  the  levels  of  characteristic  circular 
probable  errors  associated  with  both  good  and  poor  quality.  The  sampling 
procedures  and  associated  decision  criteria  must  then  be  designated. 

Once  this  is  done,  the  Gamma  distribution  can  be  used  to  derive  the 
operating  characteristics  (OC),  i.e.,  the  probability  of  accepting  a lot 
with  a specified  quality,  associated  with  the  plan.  Comparisons  of  the 
OC  of  various  plans  can  also  be  made,  and  the  best  plan  can  thereby  be 
determined. 

4.4  Development  of  Acceptance 

To  develop  the  acceptance  plans,  CPEg  is  defined  as  the  level 

of  circular  probable  error  which  represents  good  quality  of  a lot.  Poor 
lot  quality  can  be  characterized  by  any  multiple  of  CPEg,  as  long  as  the 

multiple  is  greater  than  one.  In  order  to  develop  examples  of  acceptence 
plans  it  is  assumed  in  this  report  that  poor  quality  is  characterized  by 
values  of  circular  probable  error  greater  than  2 CPEg.  One  possible 

acceptance  plan  is  to  test  e ten  round  random  sample  from  a lot  and  to 
calculate  the  observed  circular  probable  error,  CPEq,  and  then,  decide 

to  accept  or  reject  the  lot.  One  set  of  decision  criteria  Includes 
accepting  the  lot,  if  CPEq  < 1.2  CPEg  or  rejecting  the  lot,  if 

CPEq  ^1.2  CPEg.  This  plan  will  be  designated  as  Acceptance  Plan  A. 

Given  a circular  probable  error,  CPE^^,  which  characterizes  the  lot, 

CPE 

is  distributed  as  the  Gamma  distribution  discussed  in  section  4.2, 
provided  CPE^^  is  not  very  different  from  the  pooled  circular  probable 
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P(X|  <X) 


OBSERVED  CUMULATIVE 
DISTRIBUTION  OF  Xj  ' 


CUMULATIVE  DISTRIBUTION  OP  Xj  WHEN 
18  Xf  IS  DISTRIBUTED  AS  CHI-SQUARE 
WITH  18  DEGREES  OF  FREEDOM. 


0 0.30  0.60  0.90  1.20  1.60  1.80  2.10  2.40 

X {NON-DIMENSIONAL) 

Figure  2.  Cumulative  Observed  Distribution  of  Xj's  Compared 
With  That  Predicted  When  18  X;  is  Distributed  as 
Chi-Square  With  18  Degrees  of  Freedom. 


error  of  the  168  reference  round  groups.  Further  discussion  of  this 
point  Is  presented  later  In  this  section.  For  now,  It  Is  assumed  that 
CPEq 

Is  distributed  as  the  Gamma  distribution  of  section  4.2. 

1. 

To  derive  the  consumer's  and  producer's  risks  associated  with 
this  plan,  first  assume  that  CPE,  > CPEg.  Then,  based  on  the  cumulative 
Gaima  distribution  of  Figure  4, 

P(A/CPEl  ■ CPEg)  - P(CPEq  < 1.2  CPE^)  - 0.80. 

Based  on  the  cumulative  distribution  of  when  Is  distributed  as 

Chi-square  with  18  degrees  of  freedom  (Figure  2),  the  probability  of 
accepting  the  lot,  P(A),  Is  0.90.  Assuming  that  CPE|_  ■ 2CPEg,  then  based 

on  the  Gamma  distribution,  P(A/CPE|^  - 2CPEq)  ■ (PCPEq  <0.6  CPE^)  ■ 0.16. 

U-sIng  the  Chi-square  of  Figure  4,  P(A/CPEl  ■ 2CPEq)  • 0.00.  The  consumer's 

and  producer's  risks  for  this  plan  are  suimiarized  In  Table  1. 

Table  1.  Consumer's  and  Producer's  Risk  for  Acceptance  Plan  A 


Distribution  Used 

Consumer's  Risk 

Producer's  Risk 

Gamma 

0.16 

0.20 

Chi -square 

0.00 

0.10 

As  noted  In  Table  1,  the  Gamma  derived  consumer's  risk  for 
Acceptance  Plan  A Is  0.16  compared  with  the  consumer's  risk  of  0.00 
predicted  by  the  Chi-square  distribution.  Thus,  use  of  the  Chl-souare 
distribution  for  obtaining  the  consumer's  risk  for  this  plan  would 
mislead  one  Into  thinking  that  Plan  A Is  very  good  with  respect  to  the 
consumer's  risk,  while  the  Gamma  distribution  shows  that  1t  1s  not. 

Figure  5 presents  the  OC  curves  for  Plan  A based  on  the  Gamma 
and  Chi-square  distributions.  Note  that  the  OC  curve  derived  from  the 
Chi-square  distribution  crosses  the  Gamma  derived  OC  curve  at  the  point 
where  CPE^  • 1.12  CPEg.  When  CPEl  < 1.12  CPEg,  the  Chi-square  derived  OC 

curve  overestimates  the  probability  of  acceptance  the  lot,  misleading 
the  manufacturer  Into  believing  that  this  risk  Is  smaller  than  It  actually 
Is.  When  CPE|_  > 1.12  CPE.,  the  Chi-square  derived  OC  curve  underestimates 

the  probability  of  accepting  the  lot.  Therefore,  for  lots  with  poor 
circular  probable  error  characteristics  (CPE|^  > 2CPEg),  the  Chi-square 

derived  OC  curve  misleads  the  consumer  Into  believing  that  his  risk  Is 
smaller  than  It  actually  Is. 


In  Table  2 several  acceptance  plans  are  presented.  It  should 
be  noted  that  these  plans  represent  only  a finite  subset  of  an  Infinite 
set  of  acceptance  plan  strategies. 

Table  2.  Acceptance  Plan  Alternatives  B Through  H Based  On  Multiples  Of 

Ten  Round  Samples 

Acceptance  Plan 

Designation  Description  of  Acceptance  Plan 


Test  a single  ten  round  random  sample  from 
a lot.  Accept  the  lot  If  CPE^  < 1.4  CPEg. 

Otherwise  reject  the  lot. 

Test  a single  ten  round  random  sample  from 
a lot.  Accept  the  lot  If  CPE  < 1.6  CPEg. 
Otherwise  reject  the  lot.  ° 

Test  a single  ten  round  random  sample  from 
a lot.  Accept  the  lot  If  CPE  < 1.8  CPEg. 
Otherwise  reject  the  lot. 

Test  two  ten  round  random  samples  from  a 
lot.  If  the  pooled  CPE^  < 1.4  CPE^,  accept 

the  lot.  Otherwise  reject  the  lot. 

Test  a ten  round  random  sample  from  a lot. 
Accept  the  lot  If  CPE^  <1.4  CPEg.  Reject 

the  lot  If  CPEg  > 1.8  CPEg.  Otherwise,  test 

a second  ten  round  random  sample.  Then, 
accept  the  lot  If  the  pooled  CPE^  <1.4 

CPEg.  If  the  pooled  CPE^  > 1.4  CPEg,  reject  , 

the  lot. 

Test  a ten  round  random  sample  from  a lot. 
Accept  the  lot  If  CPE^  < 1.2  CPEg.  Reject 

the  lot  If  CPE^  >1.54  CPE„.  Otherwise,  test 
0 g 

a second  ten  round  random  sample.  Then,  accept 
the  lot  If  the  pooled  CPE^  < 1.2  CPEg.  If  the 

pooled  CPEg  > 1.2  CPEg,  reject  the  lot. 

Test  a tan  round  random  sample  from  a lot. 
Accept  the  lot  If  CPEg  < 1.6  CPEg.  Reject 

the  lot  If  CPEg  >2.0  CPEg.  Otherwise,  test  a 

second  ten  round  random  sample.  Then,  accept 

the  lot  If  the  pooled  CPE  < 1.6  CPE  . If 

0 g 

the  pooled  CPE  > 1.6  CPE  , reject  the  lot. 


L Operating  OKn^octeristics  of  Acceptance 


Figures  6 through  12  present  the  OC  curves  for  acceptance 
plans  B through  H described  In  Table  2.  In  each  figure,  the  consumer's 

risk  Is  determined  from  the  probability  of  accepting  the  lot  when 

CPEj^  ■ 2CPE  , When  CPE|_  ■ CPEg,  the  producer's  risk  Is  given  by  l-P(A), 

where  P(A}  is  the  probability  of  accepting  the  lot.  The  OC  curves  for 
all  of  the  plans  show  that  the  Ch1>square  derived  consumer's  and  producer's 
risks  are  lower  than  those  derived  from  the  Gamma  distribution. 

Table  3 summarizes  the  consumer's  and  producer's  risk  for 
acceptance  plans  A through  H.  The  consumer's  risks  are  presented  as 
ranges.  The  lower  bound  In  each  case  Is  obtained  from  the  Chi-square 
derived  OC  curve,  while  the  upper  bound  Is  obtained  from  the  Gamma 

derived  OC  curve.  The  producer's  risks  are  presented  as  point  estimates 

and  are  obtained  from  the  Gamma  derived  OC  curves. 

Table  3.  Consumer's  and  Producer's  Risks  for  Acceptance  Plans  A 


Acceptance  Plan 
Designation 

Through  H 

Consumer's 

Risk 

Producer's 

Risk 

A 

0.00  - 0.16 

0.20 

B 

0.04  - 0.25 

0.10 

C 

0.14  - 0.40 

0.13 

D 

0.32  - 0.50 

0.07 

E 

0.00  - 0.09 

0.07 

F 

0.03  - 0.28 

0.04 

G 

0.00  - 0.16 

o.n 

H 

0.17  - 0.55 

0.02 

CPEq 

It  was  previously  assumed  that  ||i|^  would  be  distributed  as 

the  Gamma  distribution  with  o " 7.4558  and  8 0.1261.  If  CPE|_  equals 

the  characteristic  circular  probable  error  of  the  reference  lot  used  to 
obtain  the  fitted  Gamma  distribution,  the  assumption  Is  reasonable.  The 
Gamma  distribution  was  obtained  from  the  actual  distribution  of  observed 
circular  probable  errors  over  a twelve  year  period.  It  Is  reasonable  to 
assume  that  this  Is  representative  of  the  distribution  which  will  occur 
In  the  future.  However,  as  CPE^  deviates  from  the  characteristic  circular 

probable  error  of  the  reference  lots,  the  distribution  of  CPEg  deviates 

from  the  Gainna  distribution.  The  derived  Gamma  distribution  represents 
the  deviation  in  observed  circular  probable  errors  due  to  both  Inherent 
round-to-round  differences  and  occaslon-to-occaslon  test  condition 
differences.  As  inherent  round-to-round  differences  Increase,  which  Is 
the  case  for  poor  quality  control  during  manufacturing,  they  tend  to  have 
an  Increasingly  dominating  effect  on  the  distribution  of  CPE.  relative 


to  the  effect  of  cccas1on-to<occas1on  test  condition  variation.  As  the 
Inherent  differences  increase  without  bound,  the  relative  effect  of 
occaslon-to-occaslon  variation  In  test  conditions  upon  CPEq  tends  towards 

CPE? 


zero,  and  the  distribution  of 


CPEf 


approaches  the  Chi-square.  On  the 


other  hand,  as  Inherent  round-to-round  differences  decrease,  the  scatter 
of  observed  probable  errors  Is  Increasingly  dominated  by  the  effect  of 


CPE. 

test  condition  variation,  and 

L 


win  tend  to  have  greater  variation 


than  that  predicted  by  the  Gainna  distribution.  Consequently,  as  CPE^ 


Increases  towards  poor  lot  quality,  the  true  probability  of  accepting 
the  lot  lies  somewhere  between  the  probabilities  obtained  from  the 
Chi-square  and  Gamma  derived  OC  curves  as  shown  In  Figures  5 through  12. 
As  CPE^  Improves  beyond  good  lot  quality,  the  probability  of  accepting 

the  lot  decreases  below  that  predicted  by  the  Gamma  derived  OC  curve. 

For  these  reasons,  the  consumer's  risks  In  Table  3 are  presented  as 
ranges,  with  the  Chi-square  and  Gamma  derived  risks  being  the  lower  and 
upper  bounds,  respectively.  The  producer's  risk,  since  It  Is  based  upon 
lots  of  good  quality.  Is  presented  as  a point  estimate  based  upon  the 
Gamma  derived  OC  curve.  If  CPE^  Is  approximately  equal  to  the 

characteristic  circular  probable  error  of  the  reference  rounds,  the 
Ganna  derived  risk  Is  a reasonably  good  estimate  of  the  producer's  risk. 
However,  If  CPE^  Is  better  than  the  characteristic  circular  probable 

error  of  the  reference  rounds,  the  Gamma  derived  producer's  risk  Is  a 
lower  bound.  This  could  happen  If  future  APDS  rounds  are  markedly  more 
accurate  than  the  reference  rounds. 


Reviewing  the  acceptance  plans  In  Table  3,  It  Is  evident 
that  Plan  E provides  the  best  combination  of  consumer's  and  producer's 
risks.  However,  twenty  rounds  are  always  needed  for  this  plan.  Plan  G 
Is  the  next  best  plan  and  Is  less  costly  than  Plan  E.  If  lots  are 
produced  with  CPE|_  equal  to  CPEg,  the  average  sample  size  for  this  plan 

Is  11.5.  This  1$  due  to  the  fact  that  retests  would  occur  only  15 
percent  of  the  time. 
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Figure  7 Operating 


Figure  11.  Operating  Gioracteristics 


5.  CONCLUSIONS  AND  RECOMMENDATIONS 


In  past  acceptance  tests,  fining  procedures  have  not  been  designed 
to  minimize  the  effect  of  occaslon-to-occaslon  variability  In  test 
conditions  upon  target  dispersion  characteristics.  Future  tests  should 
be  designed  to  minimize  the  time  required  to  fire  a group  of  rounds  (see 
Section  2.2  for  a detailed  discussion). 

Even  when  the  time  to  fire  a group  of  rounds  Is  minimized,  nonrandom 
dispersion  patterns  about  the  canter  of  Impact  can  occur.  In  this  case 
the  method  of  successive  differences  eliminates  the  effect  of  nonrandom 
trends  on  probable  error  estimates.  This  method  of  calculating  probable 
errors  should  be  employed  whenever  nonrandom  trends  In  shot  patterns 
occur  (see  Section  3 for  a detailed  discussion). 

Decision  criteria  for  acceptance  plans,  which  are  based  on  the  use 
of  circular  probable  error  estimates,  provide  the  optimum  utilization 
of  test  data.  Circular  probable  error  estimates  should  therefore  be 
used  as  the  basis  for  decision  criteria  In  future  acceptance  plans  (see 
Section  4.1  for  a detailed  discussion). 

cpeJ 

The  distribution  of  — §•  has  not  followad  the  Chi-square  distribution 
CPEj 

In  the  past.  The  ratio  of  observed  circular  probable  error  to  the 
characteristic  circular  probable  error  of  the  two  reference  lots 
followed  a Gamma  distribution  with  o ••  7.4B58  and  s ■ 0.1261.  This 
distribution  was  used  to  obtain  the  consumer's  and  producer's  risks  for 
several  acceptance  plans.  These  risks  were  then  compared  with  the 

CPE§ 

corresponding  consumer's  and  producer's  risks  obtained  when  — Sr 

CPEf 

follows  the  Ch1 -square  distribution.  It  was  shown  that  the  actual 
consumer's  risk  of  accepting  poor  lots  Is  greater  than  the  Ch1 -square 
derived  risk  and  less  than  the  Gamma  derived  risk.  It  was  also  shown 
that  the  Ganina  derived  producer's  risk  will  provide  a lower  bound  If 
the  dispersion  characteristics  of  future  APDS  rounds  are  superior  to 
those  of  the  reference  rounds  used  In  obtaining  the  Gamma  distribution. 
Prom  Table  3,  It  was  evident  that  acceptance  plan  E provided  the  best 
combination  of  consumer's  and  producer's  risks  (see  Section  4 for  a 
detailed  discussion). 
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_ ill  Ml  itih ilMynii  till iiiini't 


Thi  acceptance  plans  presented  In  this  report  represent  only  a small 
portion  of  the  plans  which  could  be  developed  for  APDS  type  of  ammunition. 
Undoubtedly  It  will  be  necessary  to  develop  different  acceptance  plans 
for  new  types  of  APDS  rounds  undergoing  development.  These  plans  should 
be  developed  with  recognition  that  the  OC  curves  underestimate  both  the 

consumer's  and  producer's  risks,  assuming  that  ^^^0  follows  the  Chi-square 


distribution.  The  gamma  distribution  should  be  used  for  developing  OC 
curves  for  the  various  plans.  The  consumer's  and  producer's  risks 
associated  with  each  plan  can  be  compared  and  the  best  plan  can  then  be 
selected  (see  Section  4.3  for  a detailed  discussion). 
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APPENDIX 

Horizontal  I vartlcal  and  elrpular  probable  errors  obsarvad  for  two 

I 

rafarenca  lots  of  lOStm.  APDS»  M392  rounds  on  168  occasions.  A sampla 
size  of  ten  was  used  on  each  occasion. 


Occasion 

Probable  Error  In  Mils 

Horizontal 

Vertical 

Circular 

85 

0.32 

0.14 

0.43 

86 

0.20 

0.14 

0.30 

87 

0.19 

0.18 

0.32 

88 

0.11 

0.08 

0.17 

89 

0.15 

0.11 

0.23 

90 

0.30 

0.14 

0.40 

91 

0.13 

0.18 

0.27 

92 

0.12 

0.18 

0.26 

93 

0.22 

0.19 

0.36 

94 

o.n 

0.12 

0.20 

95 

0.21 

0.16 

0.32 

96 

0.09 

0.11 

0.17 

97 

0.15 

0.22 

0.33 

98 

0.24 

0.13 

0.33 

99 

0.30 

0.17 

0.42 

100 

0.12 

0.11 

0.20 

101 

0.11 

0.11 

0.19 

102 

0.26 

0.09 

0.34 

103 

0.25 

0.15 

0.36 

104 

0.18 

0.15 

0.29 

105 

0.26 

0.17 

0.38 

106 

0.10 

0.19 

0.26 

107 

0.22 

0.16 

0.33 

108 

0.20 

0.07 

0.26 

109 

0.22 

0.32 

0.48 

110 

0.17 

0.1S 

0.28 

111 

0.20 

0.16 

0.31 

112 

0.25 

0.19 

0.38 

113 

0.30 

0.21 

0.45 

114 

0.14 

0.10 

0.21 

115 

0.18 

0.09 

0.25 

116 

0.09 

0.16 

0.22 

117 

0.15 

0.18 

0.29 

118 

0.21 

0.20 

0.35 

119 

0.20 

0.53 

0.69 

120 

0.16 

0.16 

0.28 

121 

0.21 

0.18 

0.34 

122 

0.19 

0.13 

0.28 

123 

0.11 

0.06 

0.15 

124 

0.10 

0.13 

0.20 

125 

0.12 

0.18 

0.26 

126 

0.18 

0.16 

0.29 

226 


svi 


Occasion 

Probable  Error  In  Mils 

Horizontal 

Vertical 

Circular 

127 

0.22 

0.15 

0.33 

128 

0.13 

0.11 

0.21 

129 

0.34 

0.17 

0.47 

130 

0.22 

0.27 

0.43 

131 

0.29 

0.31 

0.52 

132 

0.40 

0.17 

0.53 

133 

0.50 

0.34 

0.74 

134 

0.43 

0.32 

0.66 

135 

0.54 

0.44 

0.85 

136 

0.10 

0.17 

0.24 

137 

0.33 

0.31 

0.55 

138 

0.30 

0.27 

0.49 

139 

0.27 

0.27 

0.47 

140 

0.38 

0.14 

0.50 

141 

0.43 

0.42 

0.74 

142 

0.40 

0.31 

0.62 

143 

0.17 

0.43 

0.57 

144 

0.18 

0.14 

0.28 

145 

0.33 

0.26 

0.51 

146 

0.25 

0.14 

0.35 

147 

0.20 

0.28 

0.42 

148 

0.07 

0.18 

0.24 

149 

0.18 

0.11 

0.26 

ISO 

0.19 

0.13 

0.28 

151 

0.16 

0.23 

0.34 

152 

0.29 

0.24 

0.46 

153 

0.19 

0.24 

0.37 

154 

0.35 

0.32 

0.58 

155 

0.18 

0.14 

0.28 

156 

0.15 

0.22 

0.33 

157 

0.24 

0.08 

0.31 

158 

0.25 

0.22 

0.41 

159 

0.25 

0.15 

0.36 

160 

0.31 

0.23 

0.47 

161 

0.21 

0.14 

0.31 

162 

0.26 

0.23 

0.42 

163 

0.23 

0.37 

0.53 

164 

0.29 

0.21 

0.44 

165 

0.16 

0.23 

0.34 

166 

0.35 

0.44 

0.69 

167 

0.28 

0.18 

0.41 

168 

0.17 

0.25 

0.37 

Averago 

0.22 

0.19 

0.38 
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ABSTRACT 

An  approach  to  the  sequential  estimation  of  the  fiftieth 
percentage  point  of  a quantal  response  curve  Is  discussed.  A 
comparison  of  this  method  with  some  standard  methods,  using 
Monte  Carlo  simulation  are  presented  and  Its  ease  of  application 
and  high  efficiency  demonstrated.  The  potential  advantages  of 
this  scheme  In  the  estimation  of  the  slope  of  a response  curve 
are  also  examined. 


!•  INTRODUCTION  Experimental  Investigations  often  deal  with 
quantal  response  variables  conditioned  on  a continuous  variable. 
For  example.  In  testing  the  tolerance  to  a poison  In  a species 
of  animal,  we  test  animals  at  various  drug  levels.  Here  the 
response  variable  Y for  a given  animal  will  be  survival  (Y"0) 
or  death  (Y-1)  and  the  conditioning  variable  X will  be  dosage 
of  poison.  Thus  we  are  Interested  In  the  curve  P(Y(x)“l). 

' In  this  paper  our  primary  concern  Is  with  the  sequential 

( 

estimation  of  the  fiftieth  percentage  point  of  the  response  curve 

L denoted  L ^ and  the  slope  of  the  curve  at  x"L  Section  2 

I * ^ ^ 

I dlBcuasoB  Bome  aequentlal  and  non-Boquentlal  approaches  to  this 


problem.  Section  3 Introduces  a new  sequential  approach  related 
to  the  Mood-Dlxon  Up-Down  method  (19^8).  It's  goal  is  to  allow 
more  efficient  estimation  of  and  the  slope  of  the  curve 

for  a fixed  small  sample  size  n . The  relative  efficiency  of 
this  method  compared  to  the  Up-Down  method  and  Wetherlll's  sug- 
gested "best"  small  sample  technique  for  estimation  of  are 

examined  using  Monte  Carlo  methods  and  the  results  summarled  In 
Table  1. 

Although  our  primary  Interest  is  In  we  briefly  discuss 

estimation  of  the  p^^  percentage  pointy  Lp  for  arbitrary  p In 
Section  4. 

This  paper  represents  only  preliminary  results.  More  ex- 
tensive Investigation  of  the  relative  merits  of  the  methods  sug- 
gested here  are  required  and  underway. 

2.  STANDARD  DBS IONS  For  most  applications  the  response  curve 
Is  sigmoidal  and  two  forms  are  commonly  used  to  represent 
P{  YCx)«1  } , the  problt  form 

P{  Y(x)  ■ 1 ) - 
and  the  logit  form 


I 

i 

I 

I- 


P{  Y<x)  • 1 ) ■ {1+  exp  <-p(x-o))}"^ 

Since,  In  practice  (Finney,  1952,  p.  47),  there  is  little  to  choose 
between  the  two  forms,  we  will  limit  our  discussion  to  the  logit 
form  because  of  Its  computational  advantages. 
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Basically  our  problem  Is  to  take  a sample  (x^,y^),  (x2»y2) 
>•••  estimate  L g - o and  the  slope  of  the  response 

curve  at  x * which  Is  B In  the  logit  form.  There  are 

two  Important  aspects  to  this  problem.  First,  how  do  we  select 
the  Xj^'s  and  second,  what  estimators  to  use  for  a and  0 • 

We  are  primarily  Interested  in  the  first  aspect. 

The  standard  non-sequential  experimental  design  for  this 
case  involves  selecting  k values  of  x^  hopefully  about  a 
: and  obtaining  n^  observations  at  this  x^  . The  maximum 

liklihood  estimators  of  a and  0 are  easy  to  calculate  for 
such  a sample. 

! 

Sequential  designs  for  this  problem,  which  allow  the  choice 
of  after  observing  the  result  for  the  previous  1 obser- 

vations, allow  one  to  obtain  a specified  accuracy  in  estimation 
with  a smaller  sample  size  than  that  required  with  a non-sequent- 
ial  design.  Thus  sequential  designs  are  useful  when  testing  is 
expensive  or  when  candidates  for  entry  into  the  testing  procedure 
ai'e  scarce. 

In  oi'der  to  explore  some  of  the  difficulties  encountered 
in  sequential  designs  for  the  estimation  of  parameters  of  a 
response  curve,  we  will  briefly  discuss  some  well  known  sequential 
i designs. 

Robbins  and  Monro  (1951)  introduced  a mrthod  of  stoohastlo 
[ estimation  suitable  for  the  general  regression  situation,  and 

' Chung  (195^)  considered  the  choice  of  parameters  in  the  process 
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to  obtain  certain  asymptotically  optimum  properties*  HoJi^es 
and  Lehmann  (1953)  extended  Chimg's  work  to  cover  the  quantal 
response  problem.  If  an  ostlmate  Is  required  for  , \the 
level  of  X at  vfhlah  i'(Y(x)  ■ 1)  ■ p then  a series  of  obser- 
vations yj^(x^)  is  taken  at  levels  such  that 

*r  + 1 “ *r  “ 

where  the  a^^'s  are  positive  constants  end  after  n observations, 
^n+1  taken  as  the  estimate  of  . This  method  works  best 
for  p ■ .5  • 

Although  this  method  has  certain  asymptotically  optimal 
properties,  It  also  has  some  practical  drawbacks  when  applied 
with  small  to  moderate  sample  sizes.  First, it  Is  usually  difficult 
to  apply  the  conditioning  variable  at  arbitrary  levels  as  required 
by  this  method.  Secondly,  If  the  starting  value  for  the  process 
Is  far  from  , a substantial  bias  may  be  Introduced  In  the 
estimate. 

Another  approach  to  the  design  problem,  which  alleviates  the 
first  of  the  drawbacks  described  above,  was  Introduced  by  Dixon 
and  Mood  (1948).  The  method  Involves  a grid  of  equally  spaced 
values  of  the  conditioning  variable.  A value  x^  Is  picked  as 
close  to  as  possible  and  the  response  observed.  If  the  response 

Is  positive  at  x^  the  next  x value  Is  taken  one  step  lower 
and  If  the  response  Is  negative  at  x^  the  next  x value  Is  taken 
one  step  higher  on  the  grid.  The  same  process  Is  utilized  for 
each  subsequent  x^  . This  design  causes  the  sampled  ^o 
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cluster  about  L « thus  yielding  a more  accurate  estimate  of 

s 5 

L g whether  the  estimate  is  obtained  by  maximum  liklihood, 
minimum  chi-square  or  the  Brownlee,  Hodge  approach  (1953). 

With  the  proper  ohoice  of  starting  value  near  L g and 
step  size  near  1/3  , this  scheme  yields  a highly  efficient 
estimate  of  L 5 • However,  if  the  step  size  is  too  large 
the  efficiency  decreases' rapidly  and  if  the  starting  value  is 
chosen  too  far  from  L g significant  bias  is  introduced  with 
small  to  moderate  sample  sizes.  Thus  with  small  sample  sizes 
the  mean  square  error  of  the  estimate  will  increase  with  the  size 
of  the  step  and  the  difference  between  the  starting  value  and 
L^g  . Another  interesting  observation  noted  by  Wetherill  (1963) 
is  that  for  the  joint  maximum  liklihood  estimators  of  a ■ L g 

A 

and  3 for  the  logistic  curve,  the  efficiency  of  3 increases 
with  increased  step  size  while  as  noted  above  the  efficiency  of 

A 

a decreases.  The  schemes  presented  in  Section  3 represent  an 
attempt  to  reduce  these  difficulties. 

A modified  UP-DOWN  PROCEDURE  There  have  been  numerous  modi- 
fications of  the  up-down  method  discussed  in  the  literature,  the 
purpose  of  which  being  to  reduce  the  problems  involved  in  estimat 
ing  b g . A number  of  these  modifications  are  considered  by 
Wetherill  (1963). 

The  modification  we  present  here  is  motivated  by  a hope 
that  it  will  yield  improved  estimates  of  the  slope  3 as  well 


■V 
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as  L 5 • The  rational  for  this  scheme  Is  that  we  use  large 
steps  in  the  beginning  when  searching  for  L g and  smaller 
steps  as  we  narrow  in  on  L 5 • The  variable  step  size  makes 
the  estimate  of  L g less  sensitive  to  large  errors  in  starting 
value  and  hopefully  will  inorease  the  accuracy  of  our  estimate 
for  3 . 

A generalized  Up-Down  method  can  be  described  as  follows: 

1.  Pick  a minimum  allowable  step  size  5 . 

2.  Pick  m such  that  the  maximum  allowable  step 
size  will  be  m*3 

Z,  Pick  a starting  value  x^  . 

4,  If  the  response  at  any  value  of  the 

conditioning  variable  is  positive,  take  the 
next  X value  k^‘  6 units  lower.  If  the 
response  is  negative  take  the  next  x 
value  1^*  6 steps  higher,  where  1 kj>  < m 
and  1 i Ij^  1 w • 

This  generalized  Up-Down  method  might  bo  considered  an  Up-Down 
method  with  memory  since  kj  and  1^  at  any  step  are  functions 
of  the  entire  past  history  of  the  trial. 

In  order  to  investigate  the  effect  of  variable  step  size  on 
the  estimation  of  L g two  easily  applied  examples  of  the  above 
scheme  were  considered: 

METHOD  I 

ii 

In  Method  I,  step  size  starts  at  one  and  increases  by  one  as  | 
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long  as  the  outcomes  remain  the  same  (i.e.  all  positive  or 
negative)  up  to  a maximum  step  size  of  3.  Once  a sign  change 
occurs ( step  size  reverts  back  to  one  and  the  process  continues 
for  a predetermined  number  of  steps.  In  the  generalized  scheme 
this  would  correspond  to  defining  ms3,  k^sl^el 


^ min  [kj, +1,3]  if 
^ 1 if  y.^Bl  and 


mintli+l»3]  if 


iBl,2 


ij.  = < 1 


if  yj^*0  and  y^__^j^al 


1b1,2 , . . . ,n 


METHOD  II 


In  Method  II,  step  size  starts  at  3 and  remains  at  three  as 
long  as  the  outcomes  are  the  same.  The  step  size  is  reduced  by 
one  for  each  change  in  outcome  (down  to  a minimum  of  one)  and 
increased  by  one  whenever  the  outcome  is  the  same  as  the  previous 
one  (up  to  a maximum  of  3)  and  the  process  continues  for  a pre- 
determined number  of  steps.  In  the  generalized  scheme  this  would 
correspond  to  defining  m»3,  k^al^aS  and 


k.  = { 


minlk^tlia]  'if  V/i-l"^ 
max[k^-l,l]  if  y^-l  and 


min [1^+1, 3]  if  yi=yi.l»0 


iKl|2|*ee  |T1 


Ij  « { 


^ Max[l^-l,l]  if  yji^aO  and  y^-lal 


i~l|2yfec  iH 
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To  compare  these  two  modifications  with  the  standard  Up- 
Down  method  a Monte  Carlo  simulation  was  done  assuming  the  popu- 
lation sampled  was  logistic  with  a=0  and  3=1  . Starting 


values  of  x^sO^S  and  sample  sizes  n=34  and  9 were  con- 


sidered with  a 1000  replications  generated  for  each  combination 
of  x^  and  n . For  ease  of  computation  in  this  initial  investi- 
gation, the  following  estimator  of  due  to  Brownlee  and 

Hodges  (1953)  was  used. 


n 

.5  iBl 


n 


It  should  be  noted  that  this  estimator  although  easily  calculated 
is  asymptotically  equivalent  to  the  maximum  likllhood  estimator 


of  L.s  . Th.  bull  for  oomporlson  b.tw.en  the  modified  end 


unmodified  approaches  was  bias  and  mean  square  error.  The  results 
of  this  computer  simulation  are  presented  in  Table  1. 

The  following  observations  should  be  made  regarding  Table  1: 
1.  Th^  modified  methods  I and  II  reduce  the  bias  for 


starting  value  x^b3  , for  all  three  step  sizes 


considered  and  do  not  substantially  increase  it 


for  x^«0. 
o 


The  MSB  is  far  more  stable  with  respect  to  starting 
value  for  the  modified  methods  as  opposed  to  the 
unmodified  method.  (Recall,  this  was  one  of  the 
major  difficulties  with  the  unmodified  Up-Down 
method . ) 


>'  \ ’ 
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3.  The  modified  methods  with  step  size  = .5  are 
comparable  to  the  unmodified  method  with  step 
sizes  1.0  and  1.5  and  to  Wetherill's  recommended 
technique  (1963,  p.34).  The  MSE's  for  this 
approach  are  given  in  the  (**)  column  of  Table 

1 for  a comparable  set  of  parameters . 

Thus  the  modified  schemes  allow  estimation  of  b g , with 
small  final  step  size,  which  is  fairly  stable  with  regard  to 
starting  value,  even  for  relatively  small  sample  sizes  and  com- 
parable to  Wetherill's  "best"  approach  with  regard  to  mean  square 
error . 

4,  ESTIMATION  OF  GENERAL  Lp.  An  extension  of  the  generalized 
Up-Down  method  which  might  prove  useful  for  estimation  of  general 
Lp  can  be  described  as  follows:  Steps  1,2  and  3 are  the  same 
as  for  the  case  pa. 5 discussed  in  Section  3. 

4.  Pick  an  initial  number  of  observations 

o 

to  be  taken  at  . 

o 

5.  After  each  trial  estimate  the  proportion  p'  of 

positive  responses  at  the  level  used  for  the 

current  trial  and  consecutive  with  it,  that  is, 

back  to  the  last  change  of  level.  If  p'  > p 

and  p'  is  estimated  on  n trials  or  more 

o 

decrease  the  level  by  6 steps.  If  p'  < p 
increase  the  level  1^*  6 steps , where  0 k^  ;<  m 
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and  0 < £ m.  Both  and  are  usually 

set  equal  to  zero  if  p'  = p. 

This  scheme  is  a generalization  of  the  inverse  sampling 
rule  described  by  Wetherill  which  was  the  most  efficient  approach 
he  considered  for  the  general  problem. 

Empirical  studies  on  this  scheme  and  some  possible  alter- 
natives will  have  to  be  done  in  order  to  justify  its  usefulness. 

5 ..  CONCLUSION . The  two  special  cases  of  the  generalized  Up- 
Down  method  presented  here  are  easily  applied  sampling  schemes t 
which  go  a long  way  in  reducing  one  of  the  main  difficulties  in 
the  Up-Down  method,  that  is  the  effect  of  starting  value  on 
efficiency  and  are  themselves  compar'able  in  sfficiency  to  Wether- 
ill's  "best"  scheme  for  estimating  . 

A careful  investigation  of  othrr  special  cases  of  the  gener- 
alized scheme,  ..he  ef.fioiency  of  such  schemes  in  estimating  the 
slope  and  the  usefulness  of  thu  generalized  scneme  for  general 
Lp  are  currently  underway. 
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A MONTE  CARLO  SIMULATION  OF  A 
PROBABILITY  RATIO  SEQUENTIAL  TEST  (PRST) 
PLAN  FROM  MILITARY  STANDARD  781 C 


William  J.  Broemm 

Reliability,  Availability  and  Maintainability  Division 
U.  S.  Army  Materiel  Systems  Analysis  Activity 
Aberdeen  Proving  Ground,  Maryland  21005 


X.  INTRODUCTION 

The  topic  of  n\y  presentation  Is  a Monte  Carlo  Simulation  of 
sequental  test  plans.  The  sequential  test  plans  I am  referring  to  are 
the  Probability  Ratio  Sequential  Test  Plans*  the  PRST  Plans,  located  in 
Military  Standard  781C  (Figure  1).  Xn  the  literature,  the  PRST  plans 
coma  under  the  category  of  the  Reliability  Design  Qualification  and 
Production  Acceptance  Tests  (Exponential  DistributionK  Simply  stated, 
these  tests  deal  with  items  that  have  exponential  times  between  failures. 

I The  computer  program  discussed  in  this  presentation  it  offered  as  an 

;j  aid  to  test  planners  and  those  who  are  concerned  with  the  application  of 

the  PRST  plans.  The  utilization  of  the  methodology  proposed  should 
permit  the  test  planner,  equipped  with  a reasonable  amount  of  practical 
experience  with  the  PRST  plans,  to  make  certain  probabilistic  statements 
regarding  termination  points  in  the  plans,  namely  (l)  the  likelihood  of 
roaching  the  last  failure  and  (2)  the  likelihood  of  reaching  the  maximum 
test  time.- 

Historically,  the  PRST  plans  have  no  provision  for  estimating  the 
true  MTBF  of  an  item  of  equipment,  and  therefore,  the  total  test  time 
expected  to  be  consumed  during  testing  may  vary  significantly  (Figure  2). 
Consequently,  program  costs  and  schedules  have  to  be  planned  to  compensate 
for  this  range  of  uncertainty.  However,  with  the  help  of  the  methodology 
delineated  herein,  one  may  be  able  to  choose  an  appropriate  test  plan 
from  MIL-STD  781C,  select  a lower  test  MTBF  (e^),  select  a realistic 

range  of  true  MTBF's  for  consideration,  implement  the  simulation,  and 
finally  obtain  measures  of  the  two  likelihood  estimates  aforementioned  - 
all  of  this  in  order  to  reduce  the  range  of  uncertainty  and  therefore 
minimize  program  cost  overruns. 
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FIGUIE  2.  An  Expected  Test  Ttae  Curve 


II.  STATEMENT  OF  THE  PROBLEM 

A typical  PRST  plan  from  MIL«STD  781Ct  regardless  of  the  total 
number  of  failures  and  the  total  test  time  In  the  plan,  will  basically 
assume  an  appearance  as  depicted  in  the  following  (Figure  3).  Inherent 
in  the  ditign  of  the  PRST  arrow,  if  you  will,  are  such  things  as 
decision  risks,  a discrimination  ratio,  and  aecept-reject  criteria. 

Given  that  all  of  these  things  are  in  harmony,  of  concern  is  not  the 
general  shape  or  length  of  the  PRST  arrow  but  the  two  termination 
vectors  located  in  the  arrowhead.  For  simplicity,  we  may  designate  the 
last  failure  as  F|^  and  the  maximum  total  test  time  as  T|y|.  The  problem, 

then, becomes  the  following:  When  tasting  an  item  of  equipment  with  the 
PRST  plans,  how  often  Is  the  last  failure  reached  (Figure  4)7  In  other 
words,  what  is  the  likelihood  of  reaching  F|^  (Figure  5)7  In  a similar 

manner,  when  testing  an  item  of  eouipment  with  the  PRST  plans,  how  often 
is  the  maximum  test  time  reached  (Figure  6)7  That  is,  what  is  the  likeli- 
hood of  reaching  T^  (Figure  7)7  These  kinds  of  inquires  can  be  taken  a 

step  further  and  translated  into  probabilistic  statements  (Figure  8). 
Symbolically,  we  may  ask:  What  is  P(F|^)  and  what  is  F(T^)7 
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FIGURE  6.  Haxkui  Test  Tiae  Consideration 


III.  A DESCRIPTION  OF  THE  ALGORITHM 

Tht  purpoia  of  the  simulation,  then,  Is  to  ascertain  how  often 

and  are  reached.  The  vehicle  for  doing  this  resides  In  the  algorithm  - 

a set  of  Instructions  that  simulates  the  testing  of  an  Item  of  equipment 
using  a typical  PRST  plan  (Figure  9).  The  code,  that  Is,  set  of 
Instructions,  makes  use  of  two  program  counters.  One  counter  corresponds 
to  the  last  failure,  F|^,  and  the  other  counter  corresponds  to  the  maximum 

test  time,  T^^.  During  each  Iteration  of  the  simulation,  an  Item 

of  equipment  can  be  either  rejected,  accepted  or  put  to  further 
test  (as  exemplified  by  the  continue  test  strip).  If  the  Item  Is  rejected 
or  accepted  prior  to  reaching  truncation,  then  no  counters  are  Incremented, 
and  a new  Iteration  Is  begun.  If  the  Item  falls  within  the  continue 
strip  without  a rejection  or  an  acceptance,  then  another  failure  time  Is 
called  for,  and  the  above  process  Is  repeated. 

In  order  to  teat  against  the  maximum  test  time  value,  a particular 
failure  time  Is  compared  with  the  T^^  value  (Figure  10).  If  that 

particular  failure  time  Is  greater  than  or  equal  to  the  T^  value,  then 

the  T^  counter  la  Incremented  by  one  and  a new  Iteration  Is  begun.  If 

that  particular  failure  time  Is  leas  than  the  T|^  value,  then  another 

failure  time  Is  called  for  and  queried  In  the  same  manner. 

Similarly,  ^n  order  to  test  against  the  last  failure,  a particular 
failure  number  Is  compared  with  the  maximum  failure  number,  F|^,  to 

determine  equality.  If  equality  exists,  then  the  F|^  counter  is 

Incremented  by  one  and  a new  Iteration  Is  begun.  If  equality  does  not 
axlsti  that  Is,  If  the  particular  failure  number  Is  less  than  the  F|^ 

value,  then  another  failure  Is  called  for  and  queried  In  the  same  manner. 

Now,  If  we  allow  each  Iteration  to  be  an  Independent  event,  and  If 
we  conduct  a large  number  of  these  trials,  then  the  F|^  and  T^  counters 

can  be  shown  to  be  likelihood  estimates  for  reaching  the  last  failure 
and  the  maximum  total  test  time,  respectively.  Furthermore,  these 
likelihood  estimates  can  be  viewed  In  probabilistic  terms,  as  stated 
previously. 
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FIGURE  10,  AlgoritNi  Considerations 


IV.  A PARTICULAR  EXAMPLE  FROM  MIL-STD  781 C 


Upon  selection  of  an  appropriate  PRST  plan;  that  Is,  one  that 
Incorporates  the  desired  decision  risks,  discrimination  ratio,  and 
accept-reject  criteria,  upon  selection  df  a lower  test  MTBP  value  (6.|), 

and  upon  specification  of  a range  of  true  MTBF's  for  consideration,  one 
Is  In  a position  to  run  the  simulation  In  order  to  obtain  likelihood 
estimates  for  reaching  the  last  failure  and  the  maximum  total  test  time 
(Figure  II), 


In  order  to  demonstrate  some  of  the  results  found,  consider  Test 
Plan  III  from  MIL-STD  781C  (Figure  12).  Test  Plan  III  Is  based  on 
sixteen  (16)  failures,  decision  risks  of  10  percent  and  a discrimina- 
tion ratio  of  2.0:1.  If  the  lower  test  MTBF  Is  chosen  to  be  100  hours, 
then  the  upper  test  MTBF  will  be  200  hours.  If  we  select  a range  of 


1 


true  HTBF's  between  60  and  300  hours,  then  this  plot  (Figure  13) 


demonstrates  the  likelihood  of  reaching  the  last  failure.  Consider  an 
Item  of  equipment  with  a true  MTBF  of  60  hours.  More  than  likely  that 
Item  of  equipment  will  be  rejected  prior  to  reaching  truncation. 

Consider  an  Item  of  equipment  with  a true  MTBF  of  300  hours.  More  than 
likely  that  Item  of  equipment  will  be  accepted  prior  to  reaching  truncation. 
Clearly,  It  Is  when  an  Item  of  equipment  has  a true  MTBF  between  the 
lower  test  MTBF  value,  and  the  upper  test  MTBF  value,  Bq,  that  the 

Item  will  most  often  be  tested  to  truncation. 


similarly,  this  plot  (Figure  14)  demonstrates  the  likelihood  of 
reaching  the  maximum  total  test  time.  The  same  set  of  arguments  hold 
for  this  plot. 
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PROCEDURAL  REMARKS 
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FIOJRE  12.  Test  Plan  III  froa  MIL-STD  781C 
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FIGURE  13.  Probability  of  Reaching  the  Last  Failure 


MiL-STD  781 


Probability  of  Reaching  the  ttwlppm  Test  Tine 


V.  SUMMARY 


In  sunmary.  the  intent  of  my  presentation  has  been  to  promulgate  a 
methodology  for  determining  the  likelihood  of  reaching  the  truncation 
points  in  the  family  of  PRST  plans  (Figure  IS).  With  the  knowledge  of 
how  often  an  Item  may  be  tested  to  trunoatlon;  that  isi  (1)  the  last 
failure  In  the  plan  and  (2)  the  maximum  total  test  time  In  the  plan,  a 
test  planner  may  be  In  a better  position  to  formulate  and  execute  a 
well -conceived  test  plan  package  so  that  test  costs  and  schedules  can 
be  reduced  and  so  that  program  cost  overruns  can  be  kept  to  a minimum. 
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STATISTICAL  CONSULTING 


OR 

THERE  ARE  NO  STANDARD  PROBLEMS,  ONLY  STANDARD  SOLUTIONS 

Brian  L.  Joiner 
Statistical  Laboratory 
University  of  Wisconsin-Madlson 
1210  W.  Dayton  Street 
Madison,  Wisconsin  S37D6 

Abstract 

Statistical  consulting  Involves  solving  non-standard  problems  on  a 
time  scale  such  that  optimal  solutions  are  not  feasible  even  If  one  could 
specify  the  problem  concisely  enough.  One  must  seek  to  understand 
each  problem  properly  and  resist  the  temptation  to  slap  It  Into  the 
fold  of  an  already  existing  solution.  Here  we  give  a small  collection 
of  examples  where  "standard"  solutions,  blindly  applied  would  likely 
have  proved  worse  than  useless. 

THE  ITERATIVE  NATURE  OF  ANALYSIS 

I 

I 

Exhibit  1 Is  a schematic  of  the  key  steps  In  the  Iterative  process 
of  data  analysis  and  model  biflldlng.  The  Importance  of  the  core 
triangle  of  MODEL  IDENTIFICaIiON.  MODEL  FiniNG  and  DIAGNOSTIC  CHECKING 
has  been  forcefully  Indicated  by  George  Box  and  colleagues,  especially 
In  Box  and  Jenkins  (1?70).  Additional  key  components  Illustrated  In 
Exhibit  1 that  all  too  often  get  overlooked  In  statistical  practice  are 
the  need  to  pay  careful  attention  to  the  data  — and  Its  quality  --  and 
to  the  underlying  theory  or  structure  of  the  problem. 
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Implicit  In  Exhibit  1 Is  the  all  Important  distinction  between 
"assuming"  and  "pretending".  The  American  Heritage  Dictionary  Includes 
the  following  definitions. 

- Assume!  "To  take  for  granted,"  and 

- Pretend:  "To  play  like,  to  make  believe". 

There  Is  a subtle,  but  very  Important  distinction  between  these  two  words. 
When  doing  statistical  analysis  our  life  Is  complicated  by  the  fact 
that  we  must  continually  shift  back  and  forth  between  these  two  concepts, 
and  In  the  past  have  tended  to  rely  on  only  one  word  — assume  — • to 
describe  both. 

When  we  tentatively  define  a plausible  model  INCLUDING  THE  NATURE 
OF  THE  DISTURBANCES,  we  put  on  our  mathematicians  hat  and  ask  "What 
Is  an  optimal,  or  at  least  defensible,  way  of  fitting  this  model  to  data 
having  the  Indicated  error  structure."  To  seek  such  answers,  we  must 
ASSUME  that  we  know  the  nature  of  the  model  and  the  error  structure 
perfectly.  The  mathematics  takes  us  literally  and  treats  the  model 
as  If  It  could  "take  for  granted"  everything  we  have  said. 

Having  thus  developed  a fitting  procedure,  we  proceed  to  apply  It 
to  the  data  at  hand,  but  In  so  doing  we  switch  from  acting  as  mathematicians, 
and  turn  to  being  scientists.  As  scientists  we  can  only  PRETEND  that 
the  data  can  be  completely  described  In  the  fashion  ASSUMED  In  the 
mathematics.  Of  course,  nature  cares  not  at  all  about  our  play  acting. 

Our  pretending  does  not  change  the  underlying  model,  nor  the  error 
structure.  It  will  be  what  ever  nature  has  chosen,  and  Invariably 
nature  has  chosen  a more  complex  structure  than  that  described  by  the 
mathematics  we  have  been  able  to  accommodate. 
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We  then  move  around  the  bend  In  Exhibit  1 to  the  analysis  loop 
and  begin  DIAGNOSTIC  CHECKING.  There  our  role  Is  to  ask  If  there  are 
serious  DETECTABLE  differences  between  nature  and  what  we  PRETENDED  was 
true.  Even  while  doing  DIAGNOSTIC  CHECKING  we  need  to  switch  back  and 
forth  asking  questions  like.  IF  the  disturbances  were  uncorrelated, 
what  are  the  chances  of  observing  a first  order  autocorrelation  this 
high  or  higher. 

Perhaps  a useful  way  to  sum  this  up  Is  to  say  that  It  seems  to 
be  helpful  to  use  two  different  words  for  the  two  different  roles. 

Let  us  ASSUME  when  we're  doing  mathematics  and  let  us  PRETEND  when 
we're  fitting  models  to  data.  Using  these  two  different  words  may 
help  us  keep  from  taking  too  seriously  what  we  ordinarily  ASSUME,  but 
In  fact  can  only  PRETEND. 

Vitamin  Bo  In  turnip  greens 


Another  Important  feature  of  the  Exhibit  1 Is  the  need  for 
continued  data  checking.  The  following  two  examples  provide  useful 
Insight. 

Both  Anderson  and  Bancroft  (1959,  page  192)  and  Draper  and  Smith 

(1966,  page  229  and  339)  report  analyses  of  some  data  on  the  effect 

of  three  variables  on  the  amount  of  vitamin  B2  In  turnip  greens.  The 

2 

model  given  by  D&S  contains  three  terms  X2,  and  X2  and  gives  an 
2 

(unadjusted)  R of  90X  which  Is  somewhat  higher  than  the  75X  given  by 
A&B't  model.  Even  relatively  careful  analyses  of  thi  residuals  from 
the  D&S  model,  such  as  the  residual  plots  In  Exhibit  2,  reveals  no 
serious  problems  with  the  fitted  model. 

However,  If  one  plots  the  original  DATA  In  the  ORDER  they  were 
presented  In  the  textbook  a very  striking  pattern  ct  « be  seen.  See 
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Exhibit  2F.  The  data  drop  off  nearly  linearly.  In  fact,  a simple 
straight  line  fitted  to  the  plot  In  Exhibit  2F  gives  an  of  90%! 

What  is  the  explanation?  The  answer  Is  not  clear  and  attempts  to  get 
more  details  about  the  original  experiment  have  not  been  successful. 

^1y  conclusion  Is  that  there  was  some  other  factor  not  recorded  that  was 
the  primary  responsible  party.  It  may  have  been  that  the  values  are 
reported  In  the  order  measured  and  that  reagents  or  the  greens  themselves 
decayed  over  time.  There  Is  not  other  ready  explanation.  The  Y values 
are  not  merely  listed  In  decreasing  order  since  a number  of  Inversions 
are  apparent  In  Exhibit  2F.  The  argument  that  Xg  Is  the  most  Important 
factor  loses  credibility  on  at  least  two  accounts.  A quadratic  Is 
needed  to  fit  the  three  levels  of  Xg,  and  the  Y values  In  Exhibit  2F 
seem  to  continually  drop  off  unaffected  by  changes  In  Xg.  Careful 
data  checking  has  opened  serious  questions  about  the  quality  of  those 
data. 

Oxygen  In  steel 

Another  example  of  important  but  not  easily  detectable  time  order 
decay  of  measurements  Is  provided  In  Joiner  and  Campbell  (1976). 

(Oxygen  is  there  Inadvertently  misspecifled  as  Nitrogen.)  Two  key  plots 
are  reproduced  here  as  Exhibits  3A  and  3B.  Note  again  the  serious 
data  problems  uncovered  by  careful  checking.  In  this  case  a careful 
timely  search  for  causes  was  made,  but  no  explanation  was  found. 

At  least  the  fact  that  the  problem  existed  had  been  brought  to  light  by 
the  data  checking. 

This  leads  us  to  list  the  following  caveats. 
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"Yet  to  calculate  Is  not  in  itself  to  analyze." 

Edgar  Allen  Poe,  The  Murders  In  the  Rue  Morgue. 

"There's  less  there  than  meets  the  eye." 

Joseph  M.  Cameron. 

There  are  no  standard  problems,  only  standard 
solutions. 

Checking  residuals  Is  not  always  enough,  the  data 
must  also  be  checked. 

Using  least  squares  tells  the  mathematics  to 
make  the  residuals  orthogonal  to  all  Included  terms, 
and  to  try  to  make  them  have  a Gaussian  distribution. 

Beware  of  LURKING  VARIABLES 

- Operators  change 

- Analysts  take  breaks 

- Reagents  decay 

- Voltages  vary 

- Connections  are  made 

- and  broken 

- Etc . • etc . , etc . 

TIME  ORDER  Is  often  a proxy  for  LURKING  VARIABLES 
RANDOMIZATION  of  time  order  often  helps,  but 

- will  still  need  to  check  data,  residuals, 
for  time  order  effects,  and 

• simple  randomization  may  be  too  expensive 
In  time  consumed,  materials  used  or 
complexity  of  organization,  so 
> compromises  may  be  desirable 

Even  very  similar  plots  often  show  quite 
different  things. 

There  Is  no  substitute  for  making  a wide  variety 
of  plots. 

One  of  the  chief  benefits  of  computers  Is  the  ability 
to  make  many  plots,  easily. 

"The  standard  practice  in  which  the  mode  of  analysis 
determines  the  assumptions  about  the  data  must  be 
avoided."  David  A.  Kenny,  Psychological  Bulletin,  1975. 

"We  cannot  expect  certainty  from  data,  only  fallible 
Information.  David  A.  Kenny,  Psychological  Bulletin,  1975. 
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Using  high  powered  statistical  methods  is  seldom 
the  most  important  contribution  a good  practicing 
statistician  makes  toward  the  solution  of  a problem. 

A Non-standard  design  problem 

How  do  you  find  out  how  much  a wildlife  preserve  is  worth?  Exhibit  4 
gives  much  of  the  background  for  a study  designed  to  help  find  out. 

Randomly  selected  hunters  who  were  successful  in  obtaining  a goose 
hunting  permit  were  to  be  offered  varying  amounts  of  cash  for  their 
permit.  The  responses  from  these  actual  offers  were  to  be  compared 
with  these  from  another  group  to  which  hypothetical  questions  were 
posed  of  the  form  "If  we  were  to  offer  you  $100  for  your  permit,  would 
you  take  it?" 

The  design  problem  posed  to  us  was  how  to  distribute  the  actual 
cash  offers  over  the  range  from  $1  to  $200.  We  had  a dose-response  curve 
to  design  for  — with  the  dose  being  $.  A novel  feature  was  that  higher 
doses  were  much  more  expensive  than  lower  doses.  Existing  data  from 
hypothetical  questions  suggested  a standard  logistic  model.  If  P is 
fraction  accepting  offer,  then 

log{y^)  - o + 3log($) 

seemed  to  be  a reasonable  model.  However  there  was  reason  to  believe 
that  higher  percentages  would  accept  actual  cash  offers,  than  would 
accept  hypothetical  offers  . However,  the  magnitude  of  the  difference 
was  uiiknown. 
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The  problem  was  as  usual  non-standard.  There  were  multiple 
objectives: 

- Estimate  area  over  curve  (expected  value  of  the  resource  to  a Hunter), 

- Compare  this  area  with  that  obtained  by  two  cheaper  and 
more  standard  methods  (hypothetical  offers  and  travel  costs), 

- Compare  the  whole  curves  for  the  three  methods, 

- Compare  some  particular  points  on  the  curves. 

There  were  multiple  constraints: 

- A maximum  of  $16,000  In  offers  would  be  made, 

- We  would  lose  part  of  our  fee  for  services  If  the  actual 
payments  to  hunters  totalled  over  $12,000. 

- There  was  a fixed  cost  of  about  $5  per  offer. 

- A design  was  needed  quickly  - hunted  season  was  about  to  open. 


What  we  did 

First  we  made  several  false  starts,  each  time  helping  to  clarify  the 
problem.  Other  steps  Included: 

- Identified  "worse  plausible  case"  (many  people  accept  our  $). 

- Identified  "expected  case", 

- Under  these  and  other  scenarios  we  computed  a wide  variety 
of  things  of  Interest  for  a variety  of  designs. 

- We  selected  two  candidate  designs  to  discuss  with  researchers, 

- The  desire  to  keep  from  going  broke  became  one  primary 
consideration. 

The  somewhat  surprising  answer  was  that  the  average  selling  price 
turned  out  to  be  $160,  for  the  permit  alone.  Even  If  every  hunter  got 
his  limit  of  one  goose,  the  cost  per  pound  would  be  over  10  times  that 
In  local  stores. 
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Another  design  problem 


In  another  situation  we  sought  to  develop  procedures  for  creating 

small  fractions  of  2*3^4®5'^6®  designs  "on  the  fly".  One  such  example 

2 2 11 

was  to  create  a fraction  of  a 2356  design  in  about  60  runs.  A 
careful  search  of  the  literature  gave  little  useful  information  so  we 
developed  some  ad  hoc  procedures.  To  see  how  well  they  worked  we  tried 
them  on  a 2 3 design  with  72  runs  in  which  we  wanted  to  estimate  all 
Interactions  up  through  linear  by  quadratic.  Here  a Connor-Young  (1961) 
design  was  available  as  a benchmark. 

The  most  Important  practical  conclusion  we  reached  was  that  simple 
random  sampling  without  replacement  of  the  rows  from  the  full  design, 
gave  designs  with  about  80%  efficiency.  Since  this  was  a simple 
procedure  and  since  no  other  procedure  did  markedly  better  without  a 
lot  of  work.  It  was  decided  that  simple  random  sampling  was  good  enough. 

Literature 

Since  we  frequently  need  to  locate  articles  on  some  new  topics, 
we  make  extensive  use  of  the  Current  Index  to  Statistics.*  Volume  3 
covering  1977,  for  example,  gives  convenient  access  to  over  5000 
journal  articles,  books  and  conference  proceedings. 


* Available  from  American  Statistical  Sssociation,  806  15th  St.,  N.W., 
Washington,  D.C.  20005. 
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Teaching 


Most  students  learn  only  how  to  solve  text  book  problems—and 
worse  yet--1earn  to  think  that  all  problems  should  be  text  book  problems. 
Appendix  A reproduces  a syllabus  from  a course  designed  to  counteract 
this  shortcoming  of  our  educational  system. 

Concluding  remarks 

Those  of  us  who  practice  data  analysis  and  design  have  an 
obligation  to  keep  reminding  those  who  teach  and  do  statistical  research 
that  most  problems  do  not  fit  Into  any  standard  mold.  I hope  that  these 
brief  descriptions  help  some,  but  I know  that  more  detail  on  many  more 
problems  Is  necessary  If  the  message  Is  to  be  brought  home,  where  It  matters. 
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THEORY 


Key  steps  In  the  iterative  process  of  data  analysis  and  model  building. 
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Letters  denote  values  of  X.:  A > 0.070,  B ■ 0.020,  C - 0.474. 
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Chance  to  Shoot  a Canada  Goose 
Is  Worth  $???  to  a Hunter 


UKRAIPHOLIVK 


How  MUCK,  la  hard  eath.  la  your  hobby  worth 

Bka  to  |oIt.  would  you  ba  wtlUai  to  fora- 
go  tha  llnka  for  oua  aummar  If  aoma  myitarloui 
baaafaotor  paid  you  not  to  playT  And  if  ao.  bow 
mueb  monty  would  It  takaf 
Two  UalvaraMy  of  Wlaooiiala*Madlaon  nrofaa* 
aora  ara  trying  to  datarmina  what  financial  vaiua 
rwmtion  lu.  Thay  hava  takan  gooaa  huntarc  at 
thair  aubfact  and  ara  putting  tha  huntara  to  a taat 
'niM  piolM  It  balna  conduetad  by  Rlehard  O. 
BUOwp  aM  Thomaa  A.  HaNrlaln,  both  aaaoeiata 
profataort  in  tha  Maditon  Collaia  of  .A|r|eultttral 
and  Ufa  Se'aaeaa.  Blahop  axplainad  tha  puipota  of 
tha  raaaareh,  and  tha  way  it  la  htlng  eouduotad.  In 
atalaphonalntarvlaw. 

The  PROF1880M  obtainad  namaa  of  Wlacon* 
db  randantt  who  wara  aueeataful  in  obtaining  tha 
highly  priaad  pannlta  to  hunt  gaaiM  In  tha  Horteon 
nona  during  tha  aarly  aittlon  ihlt  yaan  Tha  aarlf 
aaaaon  o^nt  today  and  Cloaat  On.  IBi  A pamit 
andflat  a huntar  to  taka  ona  gooaa. 


Lattgn  and  chaeka  wart  aant  to  a numbtr  of 
buntan  holding  aarb  aaaaon  narmlta.  Tha  ehadta 
vary  In  amount  Bitbop  daollaad  to  ravaal  tha 
ranga.  but  ona  ptraon  who  contacted  tha  Journal 
aald  ha  had  btan  offarad  laOOi  and  anothar  IIBO, 
Each  lattar  aiid.  In  part 

"Bncloaod  la  iMa  lattar  you  will  find  a chaok 
from  tha  Uolvarilly  of  Wlaoontin,  Thit  chMk  it 
nagotldble  and  It  madt  out  In  your  nama.  Wa  art 
offtrlag  to  pay  you  thia  amount  for  fortgoing  tha 
uta  of  your  aarly  aaaaon  Horlcon  Zona  gooN  hunt* 
log  paralv . . . 

"U  you  dtddt  to  fortgo  tha  uta  of  your  parmit 
at  thla  priet.  you  muat  put  tha  parmit  and  tag  In 
tha  aneloaod  talf.addrattad  atampad  anvtiopt  and 
ratum  It  to  ut.  If  you  dadda  not  to  forago  tha  um 
of  the  parmit,  atnd  ua  tha  chack  Inatwd.  If  you 
ratum  tha  ohtek  you  ahould  kaap  your  parmit'^ 

Blahop  aald  tha  tranaactlon  wu  lagal,  btcautt 
tha  unlvaraity  will  not  uta  tha  parmit.  It  will  bi 
ratumad  whan  tha  hunting  aaaaon  la  ovar. 

Although  dapandlng  on  tha  honor  ayatam  to 
aoma  axtant  Blahop  aald.  If  anyona  triaa  to  kaap 
tha  parmit  and  alto  eaih  tha  chack,  afforta  will  ba 
maoa  to  rtoovar  tha  money. 


A SECOND  PART  of  tho  oxparlmant  Involvaa 
tandlng  n ouattionnalro  lo  othar  gooaa  huntara. 
MtmhMtnf  tha  aacond  group  aaa  not  baing  adkad 
to  giva  np  thair  parmita,  but  only  to  anawor  guaa- 
tlona,  aueb  aa  how  much  thay  anjdy  hunting,  why 
thay  do'lt,  and  how  Iona  thay  hava  Mn  intaraatad 
Jn  gooaa  hunting.  Each  naraon  who  fllla  cut  a 
quaitloanat*  willba  paid  IS. 

Altogathar  BOO  paraooa  ara  Jbatng  eontteted. 
Soma  ara  balag  aakad  to  fill  out  tha  quaaOonnalra, 
and  othcra  W iiva  up  thair  parmRa. 

Sldhop  dacUttad  to  dlaeuaa  tna  amounu  baing 
offarao  iha  huntara  to  gIva  np  thair  parmita,  aay- 
lag  ha  waa  afraid  that  knowladga  might  affaet  tha 
raaulta.  Tha  amounta  vary,  baoauaa  part  of  tha 
objaet  of  tha  raaaareh  la  to  datarmina  how  much 
monay  la  raquirad  bafora  a paraon  will  forago 
aomatmng  ho  anjoya. 


Later  in  tho  aaaaon,  tha  two  protaiaora  nlan  to 
Intarviaw  a number  of  papnio  who  do  paiflelpata 
la  tha  uuoaa  hunt.  They  wiu  aak  tha  huntarabow 
much  they  apant,  and  bow  valuable  tha  hunt  wu 
to  them  o a raerutlDnal  uporlanea.  Bbihop  and 
Hobarlain  raoalvad  a grant  of  123,000  from  tha 

Wlldllta  Managamant  Initituta  and  Raaourew  tor 
tha  Futura,  both  private  groupa  with  oftleu  In 
Wuhlngton,  D.  C. 

"Wa  ara  not  uaing  any  tax  monay,"  Blahop  aaid 
Tha  National  midlllo  Pederatlonb  Conan  va> 
tion  Directory  llatd  tha  WllWfa  Maugamant  In* 
atttuta  u a “natlomt  nonprofit,  private,  mambat- 
ahlp  organlution,  aupportad  by  Induatriu,  groupa. 
and  Individuala,  promoting  bottar  uia  of  natural 
raioureaa  for  the  welfare  of  the  Nation."  Re* 
aouiew  for  the  Future  la  llated  u an  organlutlan- 
that  "worka  to  advance  tha  davelopmant.  eonaao 
vatton,  and  un  of  natural  raaourcu  and  improve- 
ment oL  the  qudllw  of  tha  anvironmant  thraugh 
raaaareh  and  education  programa." 

Sot  all  of  the  money  la  being  uied  for  payment 
untara.  Blahop  aald  there  wara  othar  upanau. 
Including  claricu  help  and  uu  of  couputan  in 
compiling  tha  raaulta. 

Blahop  aald  tha  Wlaconain  Departmant  of  Natto 
ral  Raaouew  wu  not  Involvad  In  the  atudy,  al. 
though  dapartmam  otftdala  know  of  H.  Anthony 
Earl,  DNR  aacratary,  aid  tha  matter  had  bun 
daarad  with  hla  office,  and  that  tha  department, 
while  not  Involvad,  had  made  no  attempt  to  Intcr- 
fara. 
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APPENDIX  A 


Syllabus  for 
STATISTICAL  CONSULTING 
Fall  1978 

Instructor:  Brian  Jolnar 


BOALS  OF  COURSE 


Students  will  Ineraast  thair  knowladga  of  how  ona:  * 

A.  analyses  data; 

B.  gathers  data; 

C.  consults; 

D.  uses  the  methodology  literature; 

E.  writes  good  reports. 

GENERAL  APPBOACH 


• We  will  tackle  problems  together»  developing  needed  methodology  as 
we  go. 

• We  will  work  on: 


- old  data  sets; 

- live  consulting  data  sets. 

« old  data  gathering  situations; 

• live  consulting  data  gathering  situations. 


BENEFITS  OF  APPROACH 


• It  could  be  a very  exciting  learning  mode; 

- It  Is  well  suited  to  my  goals  for  the  course. 

CONCERNS  OF  MINE 

> It  will  be  a lot  of  continuous  work  for  me; 

- It  will  lead  Into  areas  that  I know  little  about  (could  be  embarassing) 

- Students  may  resent  me  not  "teaching*  them  anything. 

FALL  BACK  POSITION 

- Like  recent  semesters 

• Hand  out  data  sets  for  student  analyses; 

• Talk  about  relevant  methodology  In  class. 


Implications; 

• Tha  specific  techniques  covered  will  not  be  considered  as  Important 
as  the  general  problem  solving  approach. 

• Ua  must  assume  students  can  cope  with  the  literature  under  some 
guidance  from  me. 

- It  will  be  almost  like  "the  real  world”,  with  attempts  to  maximlie 
learning  opportunities  while  working  under  my  loose  supervision. 
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J MORE  DETAILS 


A.  How  to  Analyze  Data 

> start  from  questions  and  data— not  from  statistical  method; 

- check  data  carefully  and  continually; 

> formulate  goals  and  questions  precisely; 

' formulate  tentative  models 

- mechanistic 

- empirical 

- specify  analytical  approaches; 

- use  graphical  techniques  liberally 

- on  data 

- on  residuals 

> organize  hand  work; 

- use  computers  effectively*  and  intermix  with  hand  work; 

- bear  in  mind  that  summarizing  and  estimation  should  almost  always 
be  much  more  important  than  testing; 

- finding  the  real  uncertainty  Is  often  very  difficult 

- multiplicity  of  procedures  tried 

- complexity  of  structure  of  variation 

> communicating  results  is  an  integral  part  of  the  analysis 


Useful  references  (see  below  for  details): 
M&T;  T;  D&W;  D;  BH^;  M-HB;  FPP 


B.  How  to  gather  data 

- formulate  goals  precisely; 

- quantify  goals; 

- specify  variables  precisely 

- specify  classes  of  candidate  models 

- describe  blocking,  realm  of  general izability; 

- consider  alternative  means  of  exposing  true  uncertainty; 

- specify  how  you  will  seek  to  estimate 

- precision 

- bias 

- describe  plan  for  data  gathering.  In  detail; 

- describe  randomization  procedure  in  detail; 

- describe  data  logging  procedures  in  detail; 

- If  data  will  be  computerized,  describe  process; 

- specify  ways  that  data  will  be  checked; 

- specify  how  data  will  tentatively  be  explored,  summarized  and 
otherwise  analyzed; 

- communicating  alternative,  tentative  data  gathering  plans  is  an 
integral  step. 

Useful  references: 

BH^;  S&C;  Slonim;  Williams;  Kish;  S^;  C&S;  Unobtrusive;  FP?  C&C;  C. 
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C.  How  to  consult 


- how  to  Interact  productively; 

- searchina  out  the  real  problem; 

• not  limiting  oneself  to  statistics 
< helpful*  resourceful  attitude; 

> sharing  the  work; 

- proper  level  of  sophistication  of  analyses/designs; 

- communicating  your  understanding  of  problem; 

- communicating  your  intended  analysis/design  strategy; 

- communicating  results  of  your  analyses  or  plans  for  design; 

- Importance  of  good  written  communication; 


References: 


Schuchany;  Joiner  cryogenics 


D.  How  to  use  methodology  literature 


- Importance  of  not  having  to  reinvent  the  wheel  (especially 
square  ones); 

- how  to  find  references  of  Interest; 


References : 


Preface  to  CIS;  CIS;  NBS  Index 


E.  Writing  good  reports 

Good  writing  Is  a skill  that  can  and  must  be  learned.  Practice  and 
organization  are  key  Ingredients. 

Some  elements  of  a good  report  are  often: 

- executive  summary 

- description  of  problem;  motivation;  goals; 

- description  of  data  (and  listing  If  appropriate); 

- data  checking  procedures  and  results; 

- non-technical  overview  of  analysis  strategy; 

- results  of  analyses; 

- recommendations  for  Improvements  In  future  data  gathering; 

- appendix:  technical  discussion  of  analyses 

- appendix:  data  listing  (If  too  long  for  body  of  report) 

NOTE:  In  this  class  only  slxll  Inch  paper  can  be  used.  Computer  printout 
must  be  cut  or  xeroxed  to  this  size;  or  If  smaller*  may  be  taped 
firmly  to  regular  paper  like  part  of  the  text  of  your  report. 
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Some  Indicators  of  goal  achievement 

A.  How  to  analyze  data 

Reports  of  data  analysis 
< data  sets  I hand  out 

> data  from  consultations 

B.  How  to  gather  data 

Reports  of  data  gathering  plans  and  results 

- situations  handed  out 
• consultations 

C.  How  to  consult 

> diary-llka  reports  of  consultations  (see  Joiner  Cryogenics) 

- brief,  one-half  page  persona!  reactions  to  consulting  articles 

0.  How  to  use  methodology  literature 

- brief  annotated  bibliography  of  key  articles  on  some  subject  (e.g. 
errors  in  X's) 

- brief  review  of  2 recent  (appeared  In  last  six  months) 
methodology  articles 

- actual  use  of  literature  as  evidenced  In  reports  In  A,B,C  above. 

E.  How  to  write  reports 

- reports  prepared  for  A,B,C  and  D above 


Some  work  steps  In  learning  to  use  computer  effectively 
On  both  UNIVAC  1110  and  DEC  11/70  (WITS): 

- enter  data  Into  a file  and  save: 

- list  file,  edit  some  data  and  save  corrected  version; 

- delete  old  file; 

- on  UNIVAC  print  TOC  of: 

- Jol ner*0ata 

- MACC*MINITAB 

- copy  an  element  of  J0INER*DATA  or  MACC*MINITAB  and  modify 

- delete  one  row 

- change  several  values 

- save  modified  version 

- make  Minitab  runs  using  at  least  the  following  commands: 

- all  types  of  plots;  INDIC;  EXECUTE;  store  K;  LOG;  NOBRIEF; 
ONEWAY;  REGR;  analyze  residuals 

: IKItS 

- use  Mini  tab  to  enter  data  from  STJBANK  & PRINT  i PLOT 

- use  STATJOB  to  do  regression  and  regression  on  logs 

- use  STATJOB  to  do  BANKPRINT 

- transfer  data  set  from  WITS+1110 

1110>WITS 
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References: 

M-HB 

M-RM 

EDIT  manual 
STATJOB  manuals 
WITS  manuals 

To  provide  evidence  of  Mork  you  do  In  learning  to  use  the  computer: 

- keep  a diary,  and  write  a report  chronologizing  your  experience 
Including  listings  of  work  steps,  computer  programs  and  output. 


Some  bench  marks 

By  the  end  of  the  Indicated  week  each  student  should  have  handed 
In  documentation  of  the  following: 

Week  Progress 

1 - Proficiency  on  one  computer  (UNIVAC  or  DEC). 

- First  data  analysis  or  design  report. 

« - Brief  annotated  bibliography  on  some  subject. 

^ - Second  data  analysis  or  design  report. 

3 - Proficiency  on  second  computer. 

- Brief  summaries  of  two  recent  methodology  articles. 

- Brief  report  on  two  papers  on  consulting 

- Should  be  able  to  use  S&C  and  M&T  (separate  documentation 

4 not  required) 

- Ideally  should  be  Involved  as  a consultant  on  some 
project  (will  depend  on  availability  of  projects). 

- Third  data  analysis  or  design  report 

- At  least  four  more  reports  on  data  analysis  projects, 

5-15  design  projects  or  consulting  projects  of  Increasing 

complexity. 


A FEW  IMPORTANT  CONCEPTS  IN  ANALYSIS.  AND  KEY  REFERENCES 

Data  checking:  Critically  Important  but  no  really  good  references, 
See  Daniel  and  D&W  and  Pollack  & Joiner 

Transformatlons(re-expresslon):  M&T  Ch.  4-6;  ETA  Ch.  3-6;  Draper  & 
Hunter  (Tech.  1969);  S&C;  Box  & Tiao. 

Plotting  data:  M-HB;  M&T;  ETA. 

Plotting  and  other  analysis  of  residuals:  D&S;  M&T;  Anscomb  & Tukey, 
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Using  computers;  M-HB;  M-RM;  STATJOB  manuals;  BMDP  manual;  SPSS 
manual;  SAS  manual;  Rummage  manual;  other  special  purpose  programs. 

Analysis  of  variance:  Winer;  BH^;  M&T;  Kirk;  Hicks;  Scheffe;  Searle; 
MINITAB;  Rummage;  STATJOB;  BMDP  and  other  computer  packages. 

Outliers  and  seriously  heavy  tailed  data;  M&T;  Denby  & Mallows;  Outer's 
computer  program. 

Missing  Values:  (See  me  and  I'll  get  bibliography  from  Jock  E.) 

Categorical  data:  (. ..from  Camll). 

Non- Independently  distributed  observations: 

- Time  series:  B&J;  BH^ 

- Components  of  variance  situations:  Anderson  & McLean  (3  places); 

Fuller  & Battese: 

- Other  problems:  Joiner  & Campbell 

Ethics:  see  1977  CIS  2 

Mechanistic  model  building:  BH  ^ 

Empirical  model  building:  D&W;  BH^ 

Finding  the  real  uncertainty;  M&T;  BH^  „ 

Variation  In  non-l1nearly  transfonrad  functions ; Ku  paper  In  Ku  volume 

GRADING 


Each  report  (or  other  write-up)  will  be  graded  on  three  criteria 

- quality  of  work, 

- amount  of  work, 

- quality  of  exposition. 

Each  criteria  will  be  assessed  on  a three  point  scale 

+ “well  above  average  for  course, 

/ “ as  expected  (average), 

- “ below  expectation. 

At  the  end  of  the  semester  you  must  return  all  of  your  work  to  me 
so  I can  review  It  and  assign  a grade  for  the  course.  You  may  pick  up 
your  materials  after  I have  turned  In  the  grades. 

In  assigning  a final  grade  I consider  all  of  the  evidence  you  have 
submitted  during  the  semester  as  to  your  potential  as  a practicing 
statistician.  Grading  Is  roughly  as  follows: 

/\  ■ A very  fine  hiring  opportunity  for  most  anyone. 

AB  “ Definitely  above  average.  Would  be  well  suited  for  some 
Jobs  but  might  have  weaknesses  in  some  others. 
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B ■ Average  overall  perrormance.  Might  be  average  In  every  respect 
or  very  good  1n  siie  respects  with  critical  weaknesses  In 
others. 

BC  ■ Below  average.  Usually  serious  weaknesses  In  an  Important 
area*  frequently  due  to  low  productivity,  tardiness,  poor 
exposition  or  repeated  serious  undetected  ''goofs"  In  analyses 
or  designs. 

C * Even  further  below  average, 
below  C ■ ?? 


A few  comments.  I tend  to  reward  hard  creative  work  more  than  I 
punish  the  lack  of.  If  you  work  hard  and  think  a lot,  you  will  probably 
get  two  benefits i 

1.  You  will  learn  a lot,  and 

2.  You  will  get  a good  grade. 

If  you  don't  work  hard,  you  will  surely  suffer  In  the  first 
and  perhaps  In  the  second.  In  short,  you  can  gamble.  In  this  course 
(and  probably  In  others)  the  odds  are  probably  about  50-50  of  a decent 
grade  with  little  effort.  But  If  you  choose  to  gamble,  remember  the 
possible  loss  In  (2),  and  the  sure  loss  In  (1). 
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LASER  BEAM  WAR  GAMES:  DESIGN,  ANALYSIS.  AND 
MODELING  CONSIDERATIONS 

W.S.  Mallios,  R.O.  Batesole,  and  D.R.  Leal 
The  BOM  Corporation 


ABSTRACT.  Concepts  1n  classical  design,  analysis,  and  modeling  require 
reexamination  under  force  on  force  experimentation  with  real  time  casualty 
assessment.  The  degree  of  allowable  free  play  between  opposing  sides  must  be 
prespecified  In  the  design  and  conduct  of  encounters.  Severely  limiting  free 
play  can  degrade  whatever  realism  Is  achieved  through  such  games,  while  max1> 
mizing  free  play  leads  to  Imbalances  In  the  evaluation  of  treatirent  effects. 
Regarding  analysis,  adjustments  are  required  for  Imbalances.  However,  adjust- 
ments through  standard  covariance  analysis  can  be  misleading  since  treatments 
affect  many  of  the  covariables.  Regarding  development  of  simulation  models, 
a broad  class  of  cause  and  effect  relations  can  be  estimated  and  evaluated  In 
terms  of  the  experimental  data.  Methods  of  structural  regression  are  used  In 
developing  a data  based  simulation  model  for  mounted  combat. 

1.  Force  on  Force  Experimentation. 

"Hoping  to  add  a touch  of  battlefield  realism  to  Its 
peacetime  training  exercises,  the  Army  Is  developing 
a complex  laser  beam  system  that  permits  two  forces  to 
shoot  at... each  other  without  hurting  anyone...  When 
the  system  goes  Into  operation... It  will  for  the  first 
time  advance  scorekeeping  In  war  games  significantly 
beyond  the  level  of  children  shouting,  'Bang,  bang, 
you're  dead'."  Los  Angeles  Times,  8/23/76. 

In  force  on  force  experimentation,  the  trend  Is  towards  real  time  casualty 
assessment  (RTCA).  Opposing  forces  utilize  weapons  equipped  with  low  Intensity 
laser  guns,  laser-sensitive  devices  (sensors),  and  automatic  telemetric  links 
to  and  from  a computer.  When  a combatant  detects  a target  and  engages  It,  he 
fires  his  weapon  activating  the  laser  gun.  If  he  Is  on  target,  that  target's 
sensors  are  activated.  The  physical  parameters  of  the  engagement  (weapon  type, 
target  type,  target  exposure,  range,  etc.)  are  automatically  transmitted  to  the 
computer  which  records  the  data,  assesses  the  results,  and.  If  a casualty  Is 
Indicated,  transmits  that  Information  to  the  target  for  attrition. 

The  objective  of  one  such  experiment  was  to  evaluate  the  effectiveness  of 
foxhole  fortifications  In  dismounted  combat.  The  scenario  called  for  a threat 
force  to  attack  and  penetrate  defense  positions  under  a variety  of  conditions. 
Preliminary  analyses  Indicated  that  threat  tactics  --  a free  play  variable  — 
had  a major  effect  on  threat  penetration.  The  desigi.  itor  "free  play"  for  tac- 
tics means  that  a threat  team  leader  was  free  to  choi  je  tactics  he  thought  best 
In  the  particular  trial.  Composed  of  24  men  — 3 squads  of  8 men  each  — threat 
teams  commonly  employed  two  tactics  with  variations  rn  each:  (1)  two  maneuver 
squads  and  one  fire  squad,  and  (11)  one  maneuver  squod  and  two  fire  squads.  The 
latter  led  to  considerably  greater  threat  success,  a result  which  Illustrates 
that  serendipitous  effects  are  a by-product  of  free  play,  force  on  force  experi- 
ments . 
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Currently  on  the  drawing  board  is  an  experiment  planned  to  evaluate  the 
effectiveness  of  a force  guarding  a nuclear  facility  to  deter  an  attacking 
terrorist  force.  The  scenario  calls  for  a surprise  attack  by  terrorists  with 
the  Intent  of  securing  and  removing  mock  nuclear  materials  stored  on  site.  The 
perimeter  Is  brightly  Illuminated,  open  terrain.  The  terrorist  force  obscures 
Its  detection  by  attempted  entry  disguised  as  delivery  vendors,  Innocuous  trav- 
elers, or  other  deception.  Engagement  commences  upon  detection,  Identification, 
or  overt  act.  A free  play  fire  floht  ensues  until  the  terrorists  are  defeated 
or  the  terrorist  mission  Is  accomplished.  Factors  to  be  varied  In  this  scenar- 
io Include  guard-terrorist  forces  sizes,  weapons  mixes,  defense  configuration 
(elevated  guard  tower,  submerged  pill  box,  or  no  special  structure),  ard  pene- 
tration distances  from  the  outer  perimeter  to  the  storage  area. 

2.  The  Experiment  Under  Study.  Data  used  In  model  building  are  drawn 
from  TEMAks*,  an  experiment  on  the  effectiveness  of  scatterable  mines  on  an 
attacking  armor  force.  The  tactical  scenario  was  an  attack  by  a Soviet  tank 
company  through  a scatterable**,  antitank  minefield  against  a defending  U.S. 
mechanized  Infantry  team.  Simulating  the  Soviet  force  were  15  M60  tanks  rein- 
forced by  three  APC  TOWs.  One  M60  tank,  two  TOWs,  and  two  Dragons  simulated 
the  U.S.  force.  A trial  consisted  on  one  complete  attack  through  the  minefield. 
The  trial  ended  when  the  last  attacking  tank  had  penetrated  the  minefield  or 
when  all  threat  tanks  had  been  killed.  Figure  2.1  depicts  a typical  trial 
where  the  Cs  denote  the  center  of  mass,  In  successive  30  second  Increments,  of 
threat  tanks  advancing  towards  the  defense.  Minefield  dimensions  are  300  meters 
by  1000  meters.  Threat  configurations  during  minefield  entry  are  illustrated 
for  two  trials  1n  Figure  2.2.  Figure  2.3  depicts  an  end  of  trial  result  where 

M denotes  a mine  or  mobility  kill,  F denotes  a direct  fire  kill  on  a threat 
tank,  and  I denotes  a Dragon  kill  by  threat  artillery. 

3.  Statistical  Design  Considerations.  In  designing  force  on  force  ex- 
perlments,  there  are  constraints  Imposed  by  priorities,  the  budget,  time,  and 
the  state  of  the  art  of  the  experimental  technique.  The  constraints  usually 
limit  the  number  of  treatments  which  can  be  evaluated  In  the  f <per1ment.  Ac- 
cordingly, there  are  two  schools  of  thought  regarding  the  cor  luct  of  the  ex- 
periment once  treatments  are  selected: 

(3.1)  limit  free  play  by  controlling  more  of  the  uncontrolled  variables;  I.e., 
neutralize  the  effects  of  certain  uncontrolled  variables  as  opposed 

to  expanding  the  number  of  treatments.; 

(3.2)  allow  free  play  to  a maximum  possible  extent. 

Those  advocating  (3.1)  are  usually  motivated  by  Imbalances  arising  from  free 
play  --  Imbalances  which  complicate  evaluation  of  treatment  effects.  Imbalan- 
ces are  Illustrated  by  a partial  listing  of  the  TEMAWS  design  matrix  In  Table 
3.1. 


* TEMAWS  Is  an  acronym  for  tactical  effectiveness  of  scatterable  mines  In  the 
anti  armor  weapons  system. 

**  In  tactical  situations,  mines  may  be  scattered  by  artillery  or  helicopter 
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Figure  2.1  The  Advancing  Threat  Force  (excluding  reinforcements) 
Depicted  in  Terms  of  its  Center  of  Mass  (C)  in  succes 
sive  30  second  Time  Intervals. 


Figure  2.2  Threat  Configurations  During  the  30  Second  Interval 
in  Which  the  First  Threat  Tank  Enters  the  Minefield. 


'ii  HI  ' '[Hi>  = 
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Minefield  density  refers  to  the  number  of  mines  scattered  uniformly  over  an 
area  of  1000  meters  by  300  meters.  Visibility  refers  to  the  visibility  of 
the  mines  to  the  oncoming  threat  force.  Baseline  trials  are  those  for  which 
there  was  no  minefield.  Numbers  contained  within  the  cells  summarize  events 
up  to  the  time  the  first  threat  tank  enters  the  minefield.  In  comparing,  for 
example,  low  and  high  visibility  at  the  .005  density,  It  Is  seen  that  for  all 
five  trials  of  low  visibility,  at  least  one  TOW  was  killed  before  the  first 
tank  entered  the  minefield.  Thus,  for  all  trials  of  this  treatment,  the  de- 
fense was  one  weapon  short  during  the  main  part  of  the  battle  where  synergis- 
tic effects  of  the  minefield  and  defense  fire  on  threat  attrition  were  to  be 
examined.  Conversely,  for  all  six  trials  of  high  visibility,  both  TOWs  were 
available  on  first  threat  entry  Into  minefield. 

Without  adjustment  for  Imbalances,  differences  between  the  effects  of 
these  two  treatments  on  end  of  trial  dependent  variables  --  such  as  total 
threat  casualties  or  casualty  exchange  ratios  — could  be  due  to  the  differ- 
ence in  mine  visibility,  to  the  shortage  on  one  TOW,  to  a combination  of  the 
two,  or  to  other  Imbalances.  With  such  complications  brought  on  by  Imbalan- 
ces, It  Is  tempting  to  severely  limit  free  play;  e.g.,  conduct  trials  under  the 
condition  of  no  kills  on  either  side  prior  to  threat  entry  Into  the  minefield. 
The  difficulty  with  this  recourse  Is  that  It  runs  counter  to  the  purpose  of 
RTCA  — attempted  parity  with  realism.  Sacrificing  realism  to  attain  greater 
balance  Is  generally  unacceptable  since  this  recourse  elevates  the  method  of 
analysis  to  a higher  level  than  the  experimental  objectives. 

Aside  from  greater  realism,  an  additional  feature  of  (3.2)  Is  the  emer- 
gence of  serendipitous  effects.  Notice,  for  example,  the  difference  In  threat 
configurations  In  Figure  2,2.  One  configuration  shows  a single  cluster*  of 
tanks  and  the  other,  three  rather  loose  clusters.  Entry  and  passage  through 
the  minefield  In  a single,  tight  cluster  tends  to  reduce  minefield  kills  but 
Increases  direct  fire  kills;  I.e.,  a single,  tight  cluster  draws  more  defense 
fire.  Conversely,  a highly  scattered  threat  configuration,  as  auantifled  by 
a number  of  loose  clusters,  tends  to  Increase  minefield  kills  but  reduce  direct 
fire  kills.  Thus,  there  1s  a trade-off  between  minefield  kills  and  direct  fire 
kills  on  the  threat  which  Is  dependent  on  threat  clustering.  This  result  was 
not  anticipated  In  planning  the  experiment  and  would  not  have  been  uncovered 
had  the  configuration  of  threat  tanks  been  prescribed  and  nearly  constant  be- 
tween trials.  In  fact,  It  has  become  Increasingly  evident  that,  not  only  can 
serendipitous  effects  be  expected  to  occur,  but  their  effects  on  trial  outcomes 
can  be  greater  than  that  of  treatments. 

A.  Analysis  Considerations.  When  effects  of  treatments  on  end  of  trial 
dependent  variables  are  analyzed  without  adjustment  for  free  play  variables,  a 
common  result  1s  that  treatments  are  Insignificant.  This  result  can  be  mislead- 
ing since,  under  (3.2),  effects  of  free  play  variables  can  dominate  effects  of 
treatments  on  trial  outcomes;  e.g.,  the  loss  of  a defense  weapon  prior  to  threat 
minefield  entry  can  have  a greater  effect  on  trial  outcome  than  the  visibility 
of  mines.  Moreover,  adjustments  for  fiee  play  variables  In  terms  of  standard 

* Threat  force  configuration,  excluding  reinforcements.  Is  quantified  In  terms 
of  clusters  and  non  clusters  within  each  of  the  areas  prior  to  the  minefield, 
within  the  minefield,  and  beyond  the  minefield.  Clusters  are  updated  every  30 
seconds  and  are  determined  by  the  single  link  method  (1).  Within  each  area,  a 
tank  is  part  of  a cluster  If  it  1s  within  100  meters  of  a cohort.  Also  within 
each  area,  a tank  Is  a non  cluster  If  It  Is  greater  than  100  meters  from  the 
closest  cohort.  2 


covariance  analysis  are  misleading  when  treatments  affect  the  free  play  co- 
variables  and  treatment  effects  on  the  covariables  are  neglected*;  see  (4), 


An  example  of  treatments  affecting  a covarlable,  say,  the  number  of  TOWs 
remaining  on  first  threat  entry  Into  the  minefield.  Is  as  follows.  Suppose  the 
TOW  firing  rate  prior  to  threat  entry  Into  the  minefield  Is  negatively  correla- 
ted with  minefield  density.  Since  the  TOW  firing  signature  Is  pronounced,  Its 
Increased  fire  draws  Increased  threat  fire  which  decreases  the  TOW's  chance  of 
survival.  Thus,  minefield  density  has  an  Indirect  effect  on  the  number  of  TOWs 
prior  to  threat  minefield  entry  — Indirect  In  the  sense  that  tht  effect  of 
minefield  density  of  TOW  survival  Is  through  the  TOW  firing  rate  and  the  sub- 
sequent threat  firing  rate: 

TOW  Firing  Threat  Firing  Number  of  TOWs 

Minefield  - . Rate  Prior  to  + Rate  Prior  to  Remaining  on 

Density  Threat  Mine-  * Threat  Mine-  Threat  Mine- 
field Entry  field  Entry  field  Entry 

where  x-^y  reads  "x  has  a positive,  direct  effect  on  y".  When  the  flow  of 
events  are  characterized  by  this  path  diagram,  a separate  equation  must  be  con- 
sidered for  each  variable  (other  than  minefield  density)  In  evaluating  treat- 
ment effects  on  dependent  variables  of  Interest. 

The  recourse  to  situations  where  treatments  have  possible  effects  on  co- 
variables  Is  the  analysis  of  direct.  Indirect,  and  overall  treatment  effects  as 
quantified  through  a structural  regression  system;  see  Appendix  A.  In  such  an 
analysis,  each  trial  may  be  partitioned  according  to  time  Intervals,  whether 
fixed,  random,  or  a combination  thereof,  and  relevant  variables  are  measured 
from  each  of  successive  Intervals.  To  partition  according  to  random  time  In- 
tervals Is  to  partition  according  to  events.  For  example,  a scheme  for  event 
partitioning  Is  the  division  of  trials  Into  before  and  after  first  threat  en- 
try Into  the  minefield.  A irwdel  for  this  scheme  Is  as  follows: 


2]“bhh*  V *2^bhj  ’‘j  ^ *bh 
h*  i 


S^ahh*  V "5!Sh*  ^h*  S^hj  “‘j  *ah 

h*  h*  j 


where  y(,*(b)  and  y^*(9)  denote  the  h*-th  endogenous  variables  during  the  be- 
fore and  after  periods,  respectively,  h,  h*  ■ 1,...,  p;  * dy^(f)/dy^*  (f), 

the  direct  effect  of  y^wlf)  on  (T),  f * a.  b;  ■ 1;  « dy,^  (a)/dyj,*  (b), 

* Even  when  treatments  do  not  affect  covarlable,  high  correlations  between  co- 
variables  can  lead  to  poor  treatment  estimates;  see  (2),  (3). 
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th«  direct  effect  of  on  yf,{a);  the  Xj  Include  treatment  effects  which 

are  assumed  constant  within  trials;  ■ dyj^(f)/dXj,  the  direct  effect  of  Xj 
on  y^(f}:  1s  the  model  error.  In  evaluating  treatment  effects,  note  that 

Xj  may  have  direct  effects  on  both  y^(a)  and  y^(b}  and,  hence,  an  Indirect 
effect  on  y|,(a)  through  y^(b).  The  overall  Xj  effect  on  yj^(a)  Is  the  sum  of 
the  direct  and  all  Indirect  effects  of  Xj  on  y|,(a);  see  (5). 


While  simple,  model  (4,1)  has  the  disadvantage  that  variables  may  be  over 
aggregated  which  leads  to  a loss  In  Information.  For  example,  suppose  a key 
factor  In  trial  outcomes  Is  the  number  of  threat  losses  In  the  first  minute 
after  threat  minefield  entry.  Aggreoatlng  this  variable  over  the  entire  after 
period  may  obscure  Its  Importance.  As  such,  the  partitioning  of  TEMAWS  trials 
employs  both  fixed  and  random  time  Intervals.  The  random  Intervals  (or  event 
segments]  are: 

(1)  the  time  segment  to  tn,  the  time  the  first  threat  tank  enters  the 
minefield 

(11)  to  to  to  * ISO  seconds  (4.2) 

(111)  the  time  segment  beyond  to  ISO  seconds. 


The  time  line  within  each  event  segment  Is  Incremented  Into  successive  30  sec- 
ond Intervals  from  which  relevant  variables  are  measured; 

....  [to  -30,  to),  [to,  to  +30),...,  [to  +120,  to  +150),  [to  +150,  to  +180),... 

V / V ^ / 

First  Event  Segment  Second  Event  Segment  Third  Event  Segment 


A model  for  this  partitioning  Is  as  follows; 


P '■hh* 

^ ^ ®ehh+i 

h+-l  1-0 


y^w  (t-1) 


I 


J-1 


2 Shlj  ’‘j 
1*»0 


(4.3) 


where  yf,(t)  Is  the  h-th  endogenous  variable  In  the  t-th  tim'.  Interval  and 
y^(t-l)  1s  Its  1-th  lag;  (t)/dy^*  (t-1),  the  "rect  effect  of 
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y|^*(t-1)  on  y^(t)  In  event  segment  e;  e » 1,  2,  3 as  defined  In  (4.2); 

“ehhO  “ ^*'^eh  error.  Contrary  to  model  (4.1),  exogenous 

variables  (other  than  lags  of  endogenous  variables)  are  written  as  Xj(t)  to 
allow  for  variation  In  Xj  within  trials*;  • dy^^  (t)  / dXj  (t-1*),  the 
direct  effect  of  Xj  (t-1*)  on  y^  (t),  where  Xj  (t-1*)  Is  the  value  of  Xj  In 
time  Interval  t-1*. 


Treatment  effects  which  are  constant  within  trials  tend  to  have  decreasing 

direct  effects  on  y^  (t)  with  Increasing  time;  I.e.,  as  trial  time  Increases, 

treatment  effects  on  yj^  (t)  are  generally  Indirect  and  through  y^  (t-1)  and 

(t-1).  Put  simply,  as  the  encounter  progresses,  performance  variables  tend 

to  reflect  the  Initial  effects  of  treatments.  Another  reason  for  the  decreas- 
ing effect,  at  least  regarding  minefield  density,  Is  that  mines  expended  through 
encounters  are  not  replaced.  With  a large  threat  force,  the  density  would  tend 
to  decrease  with  Increasing  threat  penetration.  Effects  of  the  decreasing  den- 
sity are  reflected  through,  not  only  lagged  variables  for  mine  encounters,  but 
through  a variable  measuring  cumulative  mine  encounters. 

The  event  segments  In  (4.2)  are  chosen  such  that  coefficients  In  model 
(4.3)  remain  stable  within  segments  for  key  uncontrolled  variables**.  The  ob- 
jective Is  to  attain  stability  of  coefficients  with  as  few  event  segments  as 
possible;  I.e.,  since  coefficients  are  unstable  between  event  segments,  a 
structural  system  Is  estimated  for  each  segment.  The  30  second  Intervals  with- 
in event  segments  were  chosen  to  prevent  an  over-aggregation  of  variables  and 
to  avoid  causal  feed  backs  whenever  possible;  I.e.,  the  30  second  Intervals 
are  sufficiently  small  so  that  the  structural  system  Is  largely  recursive;  see 
Appendix  A. 

5.  Data  Based  Simulation  Models.  With  the  constant  monitoring  and  RTCA 
of  weapon  systems  throughout  the  encounter,  laser  beam  war  games  have  a major 
Impact  on  modeling:  (1)  a broader  class  of  cause  and  effect  relations***  can 
be  examined  relative  to  non-RTCA  experiments  and  (11)  cause  and  effects  rela- 
tions can  be  estimated  In  terms  of  the  experimental  data.  The  negative  side 
of  data  based  models  Is  that  there  exist  artificialities  In  field  experiments. 


* An  example  of  xj  varying  within  trials  Is  artillery  fire  which  varies  ac- 
cording to  a prespecified  rate  for  a portion  of  the  encounter. 

**  It  Is  not  feasible  to  establish  stability  of  coefficients  for  all  endogenous 
variables  due  to  their  large  number. 

***  The  term  "A  causes  B"  can  be  defined  In  terms  of  "A  affects  B",  which  Im- 
plies that  a change  In  the  level  of  B Is  associated  with  a change  In  the  level 


These,  however,  can  be  largely  mitigated  through  subjective  adjustments  of 
estimated  relations.  Thus,  at  the  very  least,  sensitivity  analyses  can  be 
performed  on  the  mitigation  of  artificialities. 

The  alternative  to  data  based  models  are  non-data  based  models  which  were 
the  only  recourse  prior  to  RTCA.  These  Include  deterministic  models  such  as 
the  original  Lanchester  moddl  (6)  and  stochastic  models  which  utilize  data  In- 
puts (7),  (8).  Generally,  these  models  do  not  or  cannot  address  a number  of 
relevant  cause  and  effect  relations.  Take,  for  example,  the  number  of  mine 
encounters  In  the  t-th  time  Interval,  say,  ME  (t).  A non-data  based  model  may 
predict  ME  (t)  conditional  on  minefield  density  and,  perhaps,  on  mine  visibil- 
ity. Under  (4.3),  the  data  based  model  predicts  ME  (t),  not  only  In  terms  of 
density  and  visibility,  but  on  the  clustering  of  tanks  In  the  mlnafield  In  t-1, 
on  the  dispersion  of  these  clusters,  on  the  rate  of  movement  of  tanks  Into  and 
out  of  the  minefield  In  t-1,  on  the  firing  rate  of  tanks  In  the  minefield  In 
t-1,  and  on  cohort  kills  among  threat  tanks  In  the  minefield  In  t-1.  Moreover, 
ME  (t)  Is  affected  by  ME  (t-l),  ME  (t-E),...;  I.e.,  the  data  based  model  does 
not  assume  that  the  present  Is  Independent  of  jihe  past  as  do  many  of  the  non- 
data based  models;  events  which  occur  at  one  time  Interval  can  have  a major 
effect  on  events  occurring  at  subsequent  time  Intervals. 

Cause  and  effect  relations  comprising  the  data  based  model  are  estimated 
In  terms  of  (4.3).  There  are  advantages  to  this  type  of  model.  Firstly,  struc- 
tural regression  Is  a well  established,  rigorous  method  of  data  based  modeling 
as  evidenced  by  Its  use  In  econometric  modeling;  see  (9),  (10).  Secondly,  when 
the  linearity  assumption  Is  Invalid  (or  when  coefficients  are  not  stable  from 
one  event  segment  to  the  next),  a separate  system  can  be  fitted  per  event  seg- 
ment; I.e.,  rather  than  having  to  convert  to  a non-linear  system,  the  time  line 
Is  partitioned  according  to  event  segments  such  that  response  surfaces  are  ade- 
quately fitted  by  hyperplanes  within  segments* *.  Thirdly,  (4.3)  Is  a vehicle 
for  evaluating  treatment  effects  under  the  scenario  of  the  experiment  from  which 
the  data  are  drawn.  This  Is  an  Important  feature  In  the  presence  of  Imbalances. 
Fourthly,  (4.3)  provides  a solid  foundation  for  excursions  to  other  scenarios 
In  evaluating  treatments  not  considered  In  the  experiment;  e.g.,  weapon  trade- 
off analyses  can  be  performed,  as  ln  Section  8;  or  "dirty  battlefield"  scenarios 
can  be  simulated  with  varying  degrees  of  obscuration  and  suppression.  Finally, 


(***cont'd)  of  A.  There  are  two  types  of  association.  Either  A Is  a mechanism 
through  which  B changes  or  A reflects  (or  Is  correlated  with)  some  unknown,  un- 
measured, unused,  or  unavailable  variable  U which  Is  a mechanism  through  which 
B changes.  Note  that  only  In  the  first  type  of  association  does  A necessarily 
preceed  B In  the  sequence  of  events.  Note  also  that  knowledge  of  the  type  of 
association  may  be  more  likelihood  than  certainty. 

* The  transition  from  one  event  segment  to  the  next  or  from  one  system  of  equa- 

tions to  the  next  Is  smooth  due  to  the  use  of  lag  variables;  I.e.,  lagged  effects 

from  the  previous  event  segment  are  used  In  the  equations  for  the  current  event 
segment. 
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this  type  of  data  based  model  provides  a means  of  quantifying  effects  of  player 
psychological  profiles  on  encounter  outcomes*. 

6*  A Path  Diagram  for  TEMAWS.  To  avoid  underidentification  In  (4.3), 
additional  information  is  required  on  a number  of  parameters  per  equation. 
Usually,  this  Information  Is  derived  from  a subjective  assessment  that  certain 
direct  effects  do  not  exist.  In  TEMAWS,  this  assessment  Is  partially  summa- 
rized by  the  path  diagram  In  Figure  6.1  wherein  all  arrows  denote  direct  effects. 
Threat  configuration  Is  t-1  affects  threat  fire,  mine  encounters,  and  defense 
fire  In  t.  In  listing  the  following  variables  which  comprise  threat  configura- 
tion, P,  W,  and  B denote,  respectively,  all  threat  positions  prior  to,  within, 
and  beyond  the  minefield: 

(6.1)  the  number  of  clusters**  and  non-clusters  In  each  of  P,  W,  and  B 
at  the  end  of  each  time  Interval 

(6.2)  minimum  ranges***  between  each  defense  weapon  and  both  clusters 
and  non-clusters  within  each  of  P,  W,  and  B 

(6.3)  the  dispersion***  of  those  clusters  In  (6.2) 

(6.4)  the  change  In  the  number  of  clusters  and  ncn-clusters  from  t-2  to 
t-1  and  from  t-3  to  t-2. 

In  model  (4.3),  these  configuration  variables  are  not  only  predictors  In  t-1, 
but  are  Individually  predicted  In  t. 

Within  t,  firings  affect  pairings,  pairings  affect  attrition,  and  attri- 
tion affects  threat  configuration.  In  turn,  configuration  In  t affects  firings 
and  mine  encounters  1,i  t+i, 


* In  linear,  reduced  models,  player  or  team  effects  are  often  quantified  In 
terms  of  dummy  variables  whose  coefficients  are  random  effects.  If,  prior  to 
the  experiment,  each  participant  responds  to  an  appropriate  psychologjal  In- 
ventory, the  responses  can  be  converted  to  scores  In  terms  of  principal  compo- 
nent or  factor  analysis;  the  scores  become  covariables  which  replace  the  dummy 
variables;  see  (1).  In  applying  model  (4.3)  to  TEMAWS  data,  player  effects  are 
reflected  by  performance  data;  I.e.,  dummy  variables  for  player  or  team  effects 
were  not  Introduced  and  no  psychological  Inventory  was  utilized  In  TEMAWS.  Mod- 
eling In  future  RTCA  experiments  could  be  enhanced  by  utilizing  an  Inventory,  by 
quantifying  players  In  terms  of  "psychological  covariables",  and  by  using  these 
covariables  as  exogenous  variables  which  are  constant  within  trials.  Proper  ro- 
tation of  personnel  allows  for  these  covariables  to  vary  between  trials. 

**  The  threat  reinforcements  are  not  Included  In  the  threat  tank  clusters. 

***  The  reciprocals  of  the  ranges  and  dispersion  are  used  In  the  model;  I.e., 
when  a cluster  or  non-cluster  Is  not  present  1n  P,  or  W,  or  B,  the  correspond- 
ing range  and  dispersion  are  set  equal  to  zero. 
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within  t,  mine  encounters  affect  mine  kills*.  Each  defense  weapon  Is  modeled 
Individually  regarding  firings,  pairings,  and  attrition**.  Threat  reinforce- 
ments and  artillery  are  modeled  Individually  regarding  firings,  pairings  and 
attrition.  For  each  t,  threat  tank  firings***  are  aggregated  and  modeled  as 
an  entity  within  each  of  P,  W,  and  B,  as  are  threat  tank  pairings  and  attri- 
tion. 


The  path  diagram  does  not  depict  lagged  effects.  Lags  of  each  variable 
extend  back  to  at  most  five  time  Intervals.  Generally,  the  lags  of  y^j^*  (t) 

do  not  have  a direct  effect  on  y^^  (t),  hi<h*.  Exceptions  to  this  are  lags 

of  mine  kills  affecting  mobility  kills  and  lags  of  pairings  affecting  firings 
for  certain  defense  weapons. 


The  length  of  the  Interval  t Is  sufficiently  small  such  that  causal  feed 
backs****  within  t are  largely  avoided.  Exception  to  this  are  feedbacks  be- 
tween firings  of  certain  defense  weapons  which  reflect  position  effects. 
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to  simulate  communications  between  defense  weapons. 

7.1  The  Method  of  Simulation.  Following  estimation  of  parameters  In  (4,3), 
encounter  outcomes  are  generated  oy  converting  predicted  values  In  t to  observed 
values  In  t which  become  predictors  for  other  variables  1n  t and  lagged  varia- 


bles for  predictions  In  t+1.  For  example,  the  predicted  number  of  firings  for  a 
given  defense  weapon  In  t Is  taken  as  the  expected  value  of  a conditional  Poisson 
distribution.  Sampling  therefrom  provides  an  observed  value  which  becomes  (1) 
the  first  lag  1n  predicting  expected  firings  In  t+1  and  (11)  a predictor  for  ex- 
pected pairings  by  this  weapon  In  t.  Observed  pairings  In  t,  obtained  analogous 
to  observed  firings,  become  (1)  the  first  lag  In  predicting  expected  pairings 
t+1  and  (11)  a predictor  for  variables  comprising  threat  attrition  In  t;  these 
variables  Include  Individual  threat  reinforcements  and  threat  tanks  aggregated 
within  each  of  P,  W,  and  B. 


Variables  not  following  conditional  Poisson  distributions,  such  as  ranges 
and  cluster  dispersions,  are  assumed  to  follow  conditional  normal  distributions. 
For  example,  the  predicted  minimum  range  between  a given  defense  weapon  and  all 
threat  clusters  within  the  minefield  becomes  the  expected  value  of  the  condition- 
al normal!  the  variance  Is  taken  as  the  estimated  variance  of  the  model  errors 
(4.3).  Sampling  from  this  distribution  provides  the  observed  value  for  this  par- 
ticular range  In  t.  This  value,  as  an  observed,  threat  configuration  variable 


* For  mine  kills  In  t,  mobility  kills  are  modeled  separately  from  mine  kills. 

**  Attrition  on  the  weapon  system  Is  modeled  separately  from  personnel  kills. 

***  Primary  and  secondary  firings  and  pairings  by  the  threat  tanks  are  mod- 
eled separately. 

****  A casual  feedback  between  two  variables  A and  B within  t,  denoted  by  A-<-^B, 
means  that  A affects  0 and  B affects  A sequentially  over  time  within  t. 


In  t,  Is  a predictor  for  both  mine  encounters  and  firings  In  t+1. 


Th<s  recursive  procedure  Is  followed  In  generating  all  predicted,  ob- 
served, and  lagged  values  for  each  (t)  In  (4.3)*.  Only  Initial  conditions**. 
In  terms  of  the  earliest  lags  In  the  first  event  segment,  must  be  prespecified. 


Simulations  are  conducted  under  the  following  ground  rules.  Regarding  end 
of  trial  criteria,  the  defense  wins  if  the  number  of  defense  weapons  remaining 
Is  oreater  than  or  equal  to  the  number  of  live  threat  tanks  plus  the  number  of 
mobility  killed  tanks.  The  threat  wins  If  the  number  of  threat  tanks  penetrat- 
ing the  minefield  1$  greater  than  three  times  the  number  of  defense  weapons  re- 
malnlno,  excluding  the  Dragons.  The  threat  also  wins  If  all  defense  weapons 
are  killed.  The  trial  Is  termed  no  decision  If  38  minutes  have  elapsed  and 
neither  side  has  won. 

The  following  constraints  are  Imposed  on  defense  firings.  After  two  fir- 
ings from  the  same  position,  one  Dragon  moves  to  a new  position;  the  other 
Dragon  moves  after  one  firing.  Movement  time  Is  three  minutes.  One  TOW  will 
assume  a new  position  after  two  firings,  the  other  after  three  firings.  Move- 
ment time  Is  two  minutes.  If  the  tank  fires  as  many  as  three  rounds  In  one 
minute,  Its  firing  Is  deterred  In  the  following  thirty  seconds***. 

The  firing  constraint  Imposed  on  the  defense  tank  reflects  obscuration  of 
Its  visibility  due  to  Its  own  firing.  The  present  simulations  do  not  account 
for  obscuration  occurring  when  a weapon  has  been  paired.  Nor  do  they  account  for 
suppressive  reactions  of  personnel  of  a paired  weapon.  Moreover,  the  ground  rules 
allow  mobility  killed  tanks  to  continue  firing.  This  creates  a dual  between 


* For  causal  feedbacks,  where  y^^  ^^^"^^^eh*  Initial,  observed  values 
^eh  ^eh*  through  the  reduced  system.  These  ob- 

served values  become,  then,  predictors  In  the  structural  system  where  y^i^  (t)-*^ 

W ^eh*  <^^-^yeh 

**  These  conditions  Include  make-up  and  numbers  of  opposing  forces.  Initial 
threat  formations,  and  Initial  ranges. 

***  Opinions  regarding  constraints  on  defense  firings  are  varied  and  will  be 
subject  to  future  sensitivity  analyses.  Due  to  the  pronounced  firing  signature 
of  both  Dragons  and  TOWs,  there  Is  general  agreement  that  each  will  relocate 
''frequently".  The  consensus  Is  that  Dragon  personnel  will  change  positions 
usually  after  one  or,  at  most,  two  firings  and  that  relocation  will  take  approx- 
imately three  minutes.  TOWs  are  said  to  relocate  after  two  or,  at  most,  three 
firings  and  that  the  relocation  time  Is  roughly  two  minutes. 
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mobility  killed  tanks  and  defensive  weapons,  particularly  for  a large  threat 
force  1n  a high  density  minefield.  The  dual  can  lead  to  unrealistic  results 
If,  in  fact,  tank  crews  are  likely  to  ball  out  of  mobility  killed  vehicles. 

It  should  be  recognized,  however,  that  constraints  can  be  Imposed  to  deal  with 
these  situations  based  on  the  foundation  provided  by  model  (4,3). 

Table  7.1  presents  selected  results  from  a simulated  trial  with  double 
the  average  number  of  threat  tanks  employed  In  TEMAWS.  The  threat  Is  declared 
a winner  after  seven  minutes  since  threat  tanks  penetrated  the  minefield  with 
a oreater  than  3 to  1 ratio;  I.e.,  seven  tanks ^penetrated  against  a remaining 
defense'  (excludlno  Dragons). of  one  tank  and  one  TOW.  Note  that  had  the  threat 
been  declared  a winner  by  penetrating  the  minefield  with  a ratio  of  at  least 
3 to  1,  the  trial  would  have  terminated  at  the  end  of  the  thirteenth  time  In- 
terval . 


8.  Simulation  Results.  TEMAWS  was  Intended  to  establish  whether  effects 
of  the  minefield  and  defense  fire  on  the  threat  are  additive  or  Interactive. 

A means  of  resolution  Is  through  the  relation  between  minefield  density  and 
the  number  of  direct  fire  kills  on  non-disabled  threat  tanks.  It  there  Is  a 
relation  between  the  two,  Effects  are  Interactive.  No  relation  Indicates  ad- 
ditivity of  effects. 

In  adjusting  for  Imbalances,  model  (4.3)  was  exercised  under  the  TEMAWS 
scenario  with  100  simulated  trials  for  densities  of  low  visibility  minefields. 
A plot  of  the  average  number  of  direct  fire  kills  on  non-disabled  tanks  versus 
minefield  density  Is  presented  In  Figure  8.1  for  force  sizes  of  13,  26,  31, 
and  39  tanks,  For  13  tanks,  the  average  number*  used  In  TEMAWS, >the  slope  Is 
negative.  Positive  slopes  result  for  other  force  sizes. 

These  slopes  can  be  explained  through  the  following  path  diagram: 


Delay 


Defense 


Minefield 

Density 


Availability 


Direct  Fire  Kills 
on 

Non-Disabled  Tanks 


A positive  effect  of  density  on  direct  fire  kills  follows  logically  from  the 
upper  path:  the  greater  the  density,  the  more  the  evasive  action  by  the  threat 
and  the  greater  the  delay  time;  greater  delays  lead  to  more  defense  fire  and, 
hence,  to  make  more  direct  fire  kills.  Positive  slopes  In  Figure  8.1  Indicate 
that  the  upper  path  dominates;  the  lower  path  dominates  with  a negative  slope.  These 


* Although  15  tanks  were  targeted  for  use,  13  were  available,  on  the  average. 


Table  7.1  A Siaulated  Trial  of  Threat  (26  Tanks.  3 APCs,  Artillery) 
Versus  Defense  (2  Dragons.  2 TOHs.  1 Tank); 

Minefield  Density  = .0005  Hines/Meter^ 
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results  Indicate  that,  given  the  force  size  In  TEMAWS,  target  availability 
prevented  the  occurrence  of  a positive  slope*.  This  Illustrates  the  utility 
of  a data  based  model  such  as  (4.3).  Excursions  can  be  made  to  encounters 
with  larger  threat  forces  so  as  to  remedy  the  problem  of  target  availability. 
The  slopes  for  these  larger  threat  forces  are  positive  Indicating  a synergistic 
effect  between  density  and  direct  fire  kills. 

Simulation  results  In  Table  8.1  summarize  encounters  of  threat  forces** 
matched  aaalnst  each  of  three  defense  forces:  the  TEMAWS  force  (2  TOWs,  2 
Dragons,  1 tank),  the  TEMAWS  force  excluding  Dragons,  and  the  TEMAWS  force 
Including  a second  tank.  It  Is  seen,  for  example,  that  a threat  force  of  13 
tanks  wins  In  331(  of  the  encounters  against  (2  TOWs,  2 Dragons,  1 tank,  den* 
sity  ■ .0005)  and  against  (2  TOWs,  0 Dragons,  1 tank,  density  ■ .001).  Since 
150  mines  were  utilized  for  a density  of  .0005,  there  Is  a trade-off  between 
the  extra  150  mines  required  to  achieve  a .001  density  and  2 Dragons.  Other 
trade-offs  are  presented  In  Table  8.2***. 


* At  the  time  of  the  TEMAWS  experiment,  a larger  number  of  tanks  could  not 
be  Instrumented  for  RTCA. 

**  Each  threat  force  size  Includes  the  same  reinforcements  as  In  TEMAWS. 

***  A word  of  caution  Is  In  order.  Ground  rules  affect  results  of  simulations. 
Results  from  a given  set  of  ground  rules  should  be  held  In  abeyance  until 
changes  In  ground  rules  are  evaluated  through  sensitivity  analyses. 
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percentage  based  on  100  sioulated  trials 


1 


Appendix  A.  A number  of  assumptions  are  usually  associated  with  model 
(4.3).  Regarding  assumed  that  E (figf,(t))  - 0 and, that 


5eh(^)  «eh*(*) 


- 0 for  t I*  t' 


• °ehh*  ^ 


(A.1) 


Evaluation  of  residual  crossproducts,  based  on  ordinary  least  squares  (OLS) 
estimation  on  a per  equation  basis  In  model  (4.3),  support  (A.l).  For  tests 

of  significance,  normality  of  distribution  Is  Imposed  on  iSg|,(1:)' 


The  lagged  variables  In  (4.3)  serve  the  purpose  of  quantifying  the  de- 
pendence of  the  present  on  the  past.  In  doing  so,  the  lags  also  reinforce 
assumption  (A.1).  Regarding  assumptions  on  the  lags,  let 

■'hh* 

S “ehh*1  ^ ^ " ®ehh*^‘’^» 

1-0 


and  let  thepxpmatrix 

It  Is  assumed  that  all  roots  of  the  equation  9g  ■ 0 are  smaller  than  1 In 
absolute  value  for  each  event  segment  e;  see  (12).  Estimated  values  of  the 
“ehh*1  assumption. 

The  structural  system  In  (4.3)  Is  rewritten  as 


(A. 2) 


where  the  p x 1 vector^  ■ (yf,(t));  the  q*  x 1 vector  z contains  all  exo- 
genous variables  Including  yj,*(t-1)  and  Xj(t-1),  1 ^ 0,  1*a0; 


q*  - q + Sruu*  + 2 
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the  p X 1 vector  ^ ■ (figj,  (t));  the  p x p matrix  ag 


«ehh* 


r 


i 


1s  the  direct  effect  of  (t)  on  (t)  In  event  segment  e;  Ogj^^  ■ 1; 
the  p X q*  matrix  contains  direct  effects  of  exogenous  variables  on  endo- 
genous variables.  The  structural  system  uniquely  determines  the  reduced 
system  and  1s  given  by 


1 ■ 

a r z + a a 
*e  e - e -e 

where 

^e 

■ • 

e 6 — e 

“o"'  ^ • 

From 

(A.l), 

ie 

5 (0.  2e6^ 

E(i,  M,)  ■ ' 

^e6  - 

^®ehh*^ 

Thus, 

! (0,  SeJ  where 

I 

^ee  e 

a;.l  . 

QLS  estimation  In  the  reduced  system  leads  to  consistent  estimates  of 

0g  and  2gj.  Application  of  OLS  estimation  In  the  structural  system  yields 

Inconsistent  estimates  unless  the  system  1s  diagonally  recursive*.  In  prac- 
tice, however,  assumptions  underlying  such  as  system  are  difficult  to  justify; 
see  (13).  As  such,  alternative  structural  estimation  techniques,  such  as  twc 
stage  least  squares  estimation**  (9),  (10),  are  applied  In  obtaining  consistent 
estimates,  assuming  Identification.  The  drawback  of  these  estimation  tech- 
niques Is  that  the  resulting  estimates  can  be  Inefficient,  especially  when 
values  of  r2  In  the  reduced  system  are  low.  For  this  reason,  OLS  estimation 
Is  applied  In  model  (4.3)  under  the  justification  that  mean  square  errors  will 
be  smaller  relative  to  other  estimation  techniques. 


* The  structural  system  1s  termei  diagonally  recursive  If  Og  Is  triangular 
and  ^gg  1s  diagonal. 

**  In  experimental  situations  where  the  structural  system  In  not  well  estab- 
lished, full  Information  estimation  techniques  are  risky;  I.e.,  a bias  In  one 
equation  Is  carried  over  to  other  equations.  As  such,  only  limited  Information 
techniques,  such  as  two  stage  least  squares  estimation,  were  considered. 
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ABSTRACT 


Reliability  Calculations  are  made  for  both  an  anti-tank  projectile 
and  a fragmentation  shell  tislng  fracture  mechanics  concepts  In  conjunction 
with  the  Monte  Carlo  method.  Reliability  estimates  are  evaluated  and 
compared  for  both  Welbull  and  Warner  stress  strength  diagram  definitions. 

A probability  density  function  representation  of  allowable  stress 
(strength)  Is  obtained  from  a fracture  mechanics  K_p  relationships  where 
specific  random  form  of  the  parameters  is  assigned.  A nomal  density 
function  Is  obtained  for  the  structural  element  stresses  by  using  results 
from  a two-dimensional  finite  element  solution. 

In  both  Welbull  and  Warner  diagram  methods  strength  density  distribu- 
tions and  parameters  are  the  results  of  laboratory  tensile  tests.  In  the 
Welbull  model  the  scale  and  shaped  parameters  were  obtained  from  the  Max- 
imum Likelihood  method.  The  Warner  diagram  method  required  a normal  and 
best  fit  density  function  for  stress  and  strength  respectively. 

Both  the  "weakest  link"  and  ssries-parallel  system  are  evaluated  for 
desirability  in  estimating  structural  reliability.  The  'Sreakest  link" 
approach  which  Introduc'S  reliability  independence  between  elements  will  thereby 
describe  a much  more  conservative  reliability  estimate  then  the  series- 
parallel  system  which  requires  at  least  two  adjacent  elements  to  fall  In 
order  to  have  structural  failure. 
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INTRODUCTION 


This  paper  describes  structural  reliability  from  an  idealized  linear 
elastic  fracture  mechanics  (F.M.)  model.  It  is  assumed  that  flaws  exist 
in  structural  elements  and  are  remote  from  any  other  flaws.  These  hypo- 
thetical flaws  are  described  by  random  variations  in  size  and  orientation. 
Although  these  assumptions  somewhat  Idealize  the  actual  flaw  distributions i 
they  do  provide  a reasonable  good  qualitative  analysis  of  failure  probabil- 
ity as  compared  to  the  conventional  deterministic  approach  which  considers 
only  one  type  of  a singularly  oriented  crack  (transverse  to  maximum  applied 
stress)  of  a determined  critical  size. 


Present  fracture  mechanics  procedures  as  applied  to  structural  relia- 
bility depend  on  determination  of  critical  crack  sizes  by  using  proper  K^. 
relati^nshlps  in  conjunction  with  known  stress  distributions  in  the  struc- 
ture ''  . Once  the  critical  crack  sizes  and  locations  are  established  a 


NUT  (Non-Destructive  Test)  procedure  is  applied  to  the  structure  in  order 
to  establish  if  this  crack  exists.  If  a critical  crack  is  located,  the 
structure  is  rejected.  It  should  be  noted  that  an  excessively  large  re- 
jection rate  can  occur  by  applying  this  methodjSince  the  probability  of 
cracks  oriented  in  this  manner  is  very  small  ^ ^ . One  of  the  primary 
objection  to  the  conventional  (F.M.)  procedure  is  the  inability  ofr^DT 
methods  to  detect  cracks  less  then  a certain  size  (e.g.  .10  inch)  . 


If  the  critical  size  computed  to  be  less  then  this  size  then  the  F.M. 
procedure  will  not  be  able  to  determine  potential  failure  in  the  structures. 


The  uncertainties  existing  in  the  use  of  the  analytic  tools  (such  as 
Finite  Element  analysis)  (F.E.)  in  obtaining  the  structural  stress  distri- 
bution can  introduce  considerable  error  in  obtaining  the  critical  flaw  size. 

For  example;  the  accuracy  of  the  F.E.  solution  depends  on  the  severity  of  stress 
gradients,  mesh  size,  types  of  elements  used  and  the  effects  of  averaging 
stresses  within  the  element. 


The  flaw  simulation  scheme  (FSS)  Introduced  in  this  paper  attempts  to 
provide  alternatives  to  conventional  methods  described  above,  the  authors 
do  not  consider  this  scheme  as  the  ultimate  answer  in  the  application  of 
F.M.  to  structural  reliability  but  rather  an  alternative.  When  more  know- 
ledge is  available  regarding  flaw  types  and  their  orientation  in  structures, 
this  method  can  provide  an  excellent  tool  for  establishing  structural  relia- 
bill  ty . 

In  using  the  Monte  Carlo  method, the  flaw  simulation  scheme  provides 
for  variations  In  crack  orientation  and  size  in  addition  to  computed  stress 
values  in  the  structure.  Crack  orientations  are  assumed  to  vary  from  0°  to 
90®  in  a uniform  random  manner.  The  crack  size  variation  Is  defined  in  ex- 
ponential functional  form  where  sizes  vary  from  a large  percent  of  very 
small  cracks  to  a relatively  small  number  of  larger  detectable  cracks[5],  Tho 
stress  values  obtained  for  cracked  structural  element  is  assumed  to  be  a 
normal  distribution  where  Coefficient  of  Variation  (CV)  is  varied  in  order 
to  determine  the  effect  of  errors  in  the  F.E.  analysis. 
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The  types  of  cracks  and  their  locations  are  the  through  center  crack, 
near  cut-out  edge,  corner  crack  and  surface  crack  Cccntcr) , The  struc- 
tural configuration  detemlnes  the  types  of  cracks  and  their  locations. 


The  model  problems  chosen  to  illustrate  the  technique  proposed  in  this 
paper  Is  the  determination  of  structure  reliability  for  an  Anti-Tank  Pro- 
jectile control  section  (fig.  1)  and  a fragmentation  shell  (fig.  2).  Both 
structures  are  made  from  relatively  high  strength  brittle  steel.  The  con- 
trol section  la  subjected  relatively  large  compressive  loads  at  the  aft 
section  and  tension  strsssea  in  the  vicinity  of  the  cut-out  region  where 
the  fins  are  attached.  The  shell  is  internally  loaded  with  a uniform 
pressure  of  14KSI.  This  load  repreaenta  a proof  test  used  in  evaluating 
shell  quality.  Reliability  determination  of  the  control  section  provided 
the  opportunity  to  evaluate  the  eeries-parallel  system  approach  because  of 
the  complex  stress  stats  Introduced  by  the  relatively  large  tension  and 
compraealve  stresaai  in  the  structure.  The  weakest  link  approach  is  more 
readily  adapted  to  the  fragmentation  shall  loading  state. 


Statistic  Evaluation  of  Variable  Strength 


The  lack  of  ductility  characteristic  of  brittle  materials  has  tv^o 
undeeirable  consequences  for  the  engineer.  Firstly  any  misfit  or  mlssel- 
ignmant  producss  local  high  stresses  which  cannot  be  relieved  by  plastic 
flow,  unlike  in  ductile  material.  Brittle  component  designs  differ  from 
those  for  similar  ductile  components  in  that  extra  attention  must  be  paid 
to  detail,  especially  in  highly  atreeaed  areas.  The  second  consequence 
is  more  fundamental;  all  materials  contain  flaws  such  as  microscopic 
cavities  end  dislocations  and,  in  loaded  brittle  materials,  these  result 
in  local  etress  concentration  within  the  material.  The  strength  of  a 
component  is  governed  by  the  chance  that  a severe  strebu  concentration  (c) 
will  be  subjected  to  a stress  (c)  such  that  the  local  stress  ca  exceeds 
the  material  strength.  The  occurrence  of  this  is  a matter  of  chance  and 
explains  the  marked  variability  generally  observed  in  brittle  mateiiol 
strengths.  It  also  explains  why  brittle  material  failures  may  start  oway 
from  the  maximum  continuum  stress;  if  no  severe  flaws  coincide  with  the 
maximum  stress,  failure  may  occur  at  a severe  flaw  subject  to  a lower 
stress  at  a position  where  ca  is  e maximum. 

To  overcome  the  strength  variability  by  drastically  reducing  the 
applied  loading  is  not  an  attractive  engineering  proposition.  What  is 
needed  Is  an  estimate  of  the  likelihood  of  failure  of  the  component  under 
a specified  load.  This  requires  a detailed  knowledge  of  the  stresses  In 
the  structure,  and  the  flaw  distribution  in  the  material.  Well  established 
techniques  are  available  for  the  stress  analysis,  some  of  which  are  mentioned 
later. 
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CROSS-SECTIONAL  VIEW  OF  FRAGMENTATION  SHELL 


The  variation  In  material  scrength  due  to  the  f'awa  In  a particular 
material  can  be  illustrated  by  fracture  tests  on  a sample  of  specimens, 

A histogram  of  the  fracture  stresses  of  both  brittle  and  ductile  material 
subjected  to  uniform  tension  Is  shown  In  Fig.  3.1;  the  frequency  of  failure 
(F.)  Is  the  fraction  of  the  sample  falling  within  the  stress  range  o to 
cr  f io.  In  the  Unit,  as  the  nunber  of  specimens  (N)  bsoomes  large,  the 
stress  Interval  (d<7}  In  fig.  3.1  can  be  reduced  to  give  a continuous  dis- 
tribution curve.  Note  the  relatively  large  variation  In  strength  of  the 
brittle  material  as  compared  to  the  corresponding  ductlles  material  of 
similar  test  spsolmsns.  Structures  with  large  variations  In  material 
strength,  as  shown  for  the  brittle  material,  require  a probabilistic 
approach  In  the  design  procedures. 

A complsmentary  form  of  fig*  3.1  la  obtained  If  the  data  is 
presented  In  terms  of  the  cumulative  failure  probability  (P .) . This 
quantity  la  the  fraction  of  the  sample  falling  at  or  below  the  stress  o; 

In  the  limit  It  is  the  integral  of  the  fraquency  distribution  with  respect 
to  stress,  l.e. , 

Pf(o)  -/Pj  do  (1) 

In  practice  the  cumulative  failure  probability  Is  usually  found  from  the 
data  using  the  "mean  ranking"  approach.  The  N failure  stresses  of  the 
sample  are  arranged  la  ascending  order;  the  cumulatlvs  failure  probability 
assoclatad  with  the  failurs  stress  in  the  list  Is, 

■ rfr 

The  probability  distribution  of  the  data  can  bn  plotted  from  this  - see 
fig.  3.2. 


Conventional  Fracture  Mechanic  Approach  to  Structural  Reliability 

Fracture  Mechanics  in  the  design  process  raqulree  the  consideration 
of  three  factors;  a stress  analysis  (F.E.),  a measure  of  fracture  toughness 
(K.J  and  the  capability  of  Inapectlng  for  cracks.  The  stress  analysis  can 
requlrs  elaborate  analysis  using  advanced  F.B.  or  simpler  closed  form  solu- 
tlone  depending  on  loading  conditions  and  structural  geometry.  Since  plana 
strain  fracture  toughness  la  a structure  sensitive  material  characteristic 
similar  to  tenalls  and  Impact  properties.  It  Is  dependent  upon  material 
condition,  strain  rate  and  temperature.  In  describing  the  fracture  tough- 
ness of  a material,  determinations  are  necessary  under  a sufficiently  wide 
variety  of  conditions  to  allow  ruallstic  assessment  of  the  minimum  value 
likely  to  be  encountered  In  design  conditions.  Some  of  the  crack  detection 
techniques  Include  ultrasonics,  dye  penetrants,  magnetic  particles  and  visual 
Inspection. 
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Analytically t the  critical  flaw  sIzq  la  defined  by  on  expression  of 
the  type  shown  below: 


a 


(3) 


where  > 

a ■ critical  flaw  depth. 

C 

q ■ a parameter  which  takes  into  account  the  ahapa  of  the  flaw. 

■ plane  attain  fracture  toughness  of  the  material. 

0 ■ the  tensile  component  of  atrese  acting  normal  to  the  plane 
of  the  flaw. 

The  depth  a and  orientation  of  flaws  which,  on  the  basis  of  the 
fracture  toughneaa  and  streae  factors,  can  result  in  catastrophic  crack 
propagation.  It  should  be  noted  that  a as  defined  above  aaeumea  flaw  la 
normal  to  acting  stress  (see  figure  4).  This  assumption  rejects  the  pos- 
sibility that  flaw  could  be  oriented  in  other  directions,  thereby  neglect- 
ing obvious  possibilities  in  favor  of  an  unlikely  one.  This  could  result 
in  Incorrectly  determining  critical  flaws  slse  a . Present  flaw  detection 
methoda  in  many  instances  are  not  capable  of  detecting  critical  flaws  of  a 
relatively  small  alee.  In  factr  comments  by  Indicate  flaws  less  then 
•1  inch  of  the  time  cannot  be  found  in  a structure.  Of  course  tha  other 
difficulty  involves  not  finding  a detectable  flaw  size  In  a structure 
although  it  does  exist. 


Flaw  Simulation  Method 

An  alternative  to  the  previously  described  conventional  Fracture 
Mechanics  approach  is  made  by  Introducing  variation  In  crack  orientation 
and  length  by  means  of  the  Monte  Carlo  method.  Initially  four  typos  of 
cracktt  are  to  be  considered  in  an  element  (see  fig.  5).  The  type  and 
location  of  cracks  depends  on  the  structural  configuration. 

In  the  simulation  scheme  the  allowable  stress  o from  relation- 
ships is  written  as:  c ic 


o 

c 


f(Kjj,,  1,  6) 


(4) 


where: 

I ■ crack  length. 

0 " angle  of  Inclination  of  crack. 
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0.8  • O'  0-*  0.8  • 


ASSUMED  IN  SIMULATION  SCHEME 


It  Is  asauned  that  a represents  the  material  strengths  and  depends  on 
the  parameters  K_-,  £,  6.  The  variations  in  is  represented  by  a 
normal  probability  density  function  (P.D.F)  (fig.  6a).  Tlie  angle  9 is 
represented  by  uniform  random  numbers  in  range  of  0 to  90*  (fig.  6b). 
The  distribution  of  sizes  i is  of  an  exponential  F.D.F.  form  shown  in 
fig.  6c.  The  0 distribution  is  obtained  from  generating  a set  of 
uniform  random  Suadiera  and  solving  for  x in  the  relation. 


R, 


(5) 


where  S uniform  random  number  and  f . corresponds  to  the  desired  type  of 
frequency  distribution.  The  K-^  distribution  requires  test  results  for 
material  used  in  the  structure'^i'n  order  to  obtain  the  necessary  mean  and  stan- 
dard deviation  valuea.  In  figure  6c  the  maximum  crack  length  i.  repre- 
sented by  the  amallaat  detectable  crack  consistent  with  the  capability  of 
present  available  NDT  methods.  The  assumed  exponential  form  has  been  sub- 
stantiated by  [5,6]  in  laboratory  tests. 


A PDF  can  be  obtained  for  the  allowable  stress  a by  randomly  selecting 
from  K-q,  6 and  i distributions  discreet  sets  of  numbirs  and  subetltutlng 
them  Inco  equation  4.  Note,  there  should  be  an  equal  amount  of  say  N random 
numbers  for  each  parameter  in  order  to  have  N numbers  representing  the  a 
distribution.  ^ 


The  relationships  for  are  written  asi 
(a)  Through  crack 


0^  - Kj.j,(2/iT)^^^(i  sin*)"^^^ 

where  ■ Kj  + Kj.j,, 

Kj  ■ 0(lff/2)^^^8in^^  and 
Kij  ■ 0(iTr/2)^^^8in9co8  0. 


(b)  Comer  crack 


0^  - 2/Kj.^(1.28)(1/it)^^^ 


(7) 
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(c)  Surface  crack 


where  Q - E(K)^  - .212{a^  (8) 

y® 

- 1 - 4(a/£)^ 

0 < a < .10  Inches 
and 

0 < 1 < .05  Inches. 

In  order  to  use  Ocfy^latlonshlp  for  the  inclined  edge  crack  a solu- 
tion was  obtained  from  where  a Modified  Mapping  Collocation  scheme 
was  used.  The  results  are  tabulated  in  Table  1.  The  appropriate  inter- 
polation procedure  was  applied  in  order  to  use  Monte  Carlo  method  as 
outlined  in  equation  4. 


Structural  Stress  Analysis 


In  order  to  obtain  the  stress  distributions  in  the  Anti-Tank  projectile 
(ATP)  and  fragmentation  shell  a finite  element  (P.E.)  method  was  applied. 

As  described  previously  the  loading  consists  of  a set-back  type  compressive 
load  acting  at  a base  of  ATP  and  a Internal  pressure  proof  test  load  applied  to 
the  shell.  Ractanmdar  elements  were  used  in  the  analysis  for  both  struc- 
tures where  shell  and  ATP  contains  693  and  601  elements  respectlvelv.  The  FiE. 
solution  determines  the  average  maximum  and  minimum  principal  stresses  in 
each  element.  The  maximum  stress  is  used  in  the  reliability  determinations. 
These  stresses  should  not  be  confused  with  critical  stresses  obtained  from 
the  relationships  previously  described.  Having  availability  of  element 
stresses  and  corresponding  a or  strength  values  the  reliability  of  the 
elements  can  be  determined. 


1*  Element  Reliability  Calculation  Methods 

Element  Reliability  as  related  to  the  stress-strength  (Warner)  diagram*-®^ 
method  (sed  Appendix  A)  assumes  that  the  probability  of  survival  (Reliabil- 
ity) is  the  probability  that  material  strength  will  be  greater  then  the 
stress  in  a given  structural  element  over  a range  of  stress  values.  The 
uncertainties  in  the.  F.E.  solution  are  represented  by  a normal  distribution 
fj,  where  calculated  moan  stress  and  assumed  C.V.  are  the  functional  para- 
meters, The  distribution  is  obtained  from  known  strength  data  (e.g. 
laboratory  testa).  This  distribution  does  not  necessarily  have  to  be  a 
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inclined  Edge  Cracks  (60%) 


^ Hpi  ■ Ki/t^rT  Hq  ■ Kn/Zirr 


0 

0.000 

0.000 

10 

.160 

.170 

20 

.296 

.280 

30 

.461 

.335 

40 

.680 

.372 

45 

.705 

.365 

50 

.781 

.354 

60 

.920 

.305 

70 

1.028 

.224 

80 

1.098 

.118 

90 

1.124 

.000 

«^c 

■ K|c/H|^v^ 

Table  1 


normal  function.  It  can  be  any  P.D.F  that  accurately  represents  the 
empirical  ranked  data.  In  Appendix  A,  the  probability  of  say  s occurring 
is  fjds  while  the  probability  of  strength  greater  then  S,  is  repre- 
sented by  the  Integral*  with  fj^  Integrand  and  llfnlta  of  Sj  and  Multi- 
plication of  these  elements  provides  the  necessary  independence  between 
the  two  conditions*  Finally,  integration  over  entire  range  of  stress 
values  defines  prob^llity  of  survival  of  each  element* 

Element  reliability  niunbera  were  obtained  from  a approach  similar 
to  the  one  previously  described  except  that  discreet  values  obtained  from 
the  Monte  Carlo  simulation  ware  used  to  represent  both  and  element 
stress  density  functions.  This  Is  a reasonabla  approac!.  since  distribution 
of  strength  values  do  not  necessarily  conform  to  any  known  density  function. 
The  tails  of  the  two  density  functions  are  also  more  accurately  represented 
then  by  some  crude  approximating  function.  Thla  method  la  outlined  In 
Appendix  B where  the  probability  of  element  survival  Is  defined  as  follows  ; 
^ 1,  when  strength  is  greater  then  a tress  values  otherwise  it  la  zero. 

This  process  is  completed  whan  all  cotsbinations  are  considered.  The  re- 
lationship Pa^  therefore  defines  the  Kth  element  reliability  number. 

Since  the  Weibull  function  is  well  adopted  to  brittle  material  aub- 
Jectad  to  uniform  tenaion  state  It  was  Included  for  comparison  purposes 
in  the  probability  of  survival  calculations  of  the  fragmentation  shell. 

The  Weibull  P.D.F.  is  common  used  in  ceramlca  and  other  brittle  materials 
evaluation.  It  uaea  the  "weakaat'  link"  concept  which  is  consistent  with 
failure  phenomenon  of  brittle  materials  which  are  subjected  to  tensile 
straaaea  primarily.  A plot  of  strength  vs  cumulative  density  function 
(C.D.F.)  for  HFl  steel  ueed  In  the  fragmentation  shell  construction  Is 
shown  In  fig,  7.  Note  the  excellent  correlation  between  emperlcal  data 
and  the  Weibull  functionl^]. 


The  Weibull  probability  of  survival  Ps.  for  individual  stressed 
components  Is  written  as, 


m 


where  K « 1 for  simple  tensile  stress,  ■ volume  of  element,  V*  ••  volume 
of  test  specimen,  omax^  ■ maximum  principal  stress  In  the  element,  o and 
m are  functional  parameters  obtained  from  test  data  using  the  maximum 
likelihood  method. 

It  Is  obvious  from  equation  9 that  Fg  is  functionally  volume  dependent. 
That  is,  larger  volume  smaller  P numbers. 

® J 
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Structural  Reliability 

In  order  to  obtain  F of  the  entire  structure  the  weakest  link 
concept  is  applied • that  Is  It  asaumad  that  each  event  or  probability 
of  survival  of  elenant  la  Independent  of  any  other  one  In  the  structure. 
Therefore  the  total  la  written  aa 

N 

p.  - n P8„,  (10) 

• K-i  *• 

whara  N number  of  elemente  In  the  structure  and  Ps„  la  the  Probability 
of  Survival  of  the  Individual  element.  The  corresponding  probability  of 
failure  la  defined  aa  F^  ■ 1 - P^. 

A aerlea-parallal  approach  la  Introduced  In  order  to  examine  the 
case  where  more  then  one  clement  la  required  to  failure  In  order  to  have 
total  atructural  failure.  This  method  la  deacrlbad  by  examining  a four 
element  structure  where  two  elements  muat  fall  (see  Appendix  C) . The  F^i . 
values  are  determined  for  the  elementa  by  one  of  the  msthode  previously 
described.  The  resultant  P for  the  four  elements  ie  determined  from 
application  of  the  series  approach.  It  should  be  noted  that  this  method 
la  aomawhat  less  conservative  then  the  conventional  weakest  link  method. 
It  la  possibly  mors  rsallatlct  especially  so  for  the  complex  itatas  that 
exists  In  the  ATP  structure. 


Numerical  Results  and  Dlecuaelon 

Numerical  reliability  (R)  results  are  tabulated  In  Table  II  for  the 
fragmentation  ahell  subjected  to  an  Internal  pressure  of  14KSI  (Proof  test 
load)  as  a function  of  minumum  datectable  crack  size.  The  Warner  Diagram 
method  aa  outlined  In  the  text  le  represented  by  a normal  stress-strength 
P.D.F.  determined  from  F.E.  solution  and  laboratory  strength  tests  results 
raapactlvaly.  The  Monts  Carlo  Fracture  Mechanics  (F.S.S.)  results  are 
obtained  from  applying  the  schema  presented  In  the  text  where  both  stress 
and  strength  P.D.F.  are  the  results  of  using  the  simulation  scheme  described 
In  Appendix  B.  The  Welbull  R values  were  obtained  from  application  of 
equation  9.  It  should  be  noted  that  both  Warner  and  Welbull  methods  do  not 
shew  variation  dua  to  changes  in  cracks  sizes,  which  le  expected. 

Results  from  the  Monte  Carlo  method  Indicate  the  Importance  of  finding 
a crack  of  at  least  .025  Inches  or  leas  in  order  to  establish  at  least  92% 
probability  of  survival.  The  Warner  Diagram  and  WellMll  methods  show  much 
less  conservative  estimate  of  1 and  22  failures  In  ICiO  respectively.  Since 
Warner  and  Welbull  methods  do  not  consider  (F.M.),  thn  R independency  due  to 
crack  size  is  expected.  Although  it  is  assumed  that  tracks  do  exist  in  each 
element  and  the  weakest  link  argument  le  used,  a goou  qualitative  measure  of 
assessing  the  importance  of  finding  cracks  of  a relatively  small  size  is  ) 


available!  Coefficient  of  Variation  (C.V.)  of  10%  represents  the  variation 
in  element  stress  values  obtained  from  the  finite  element  solution. 

In  table  III  Probability  of  survival  estimates  are  for  variation  in 
creek  size  and  element  stress  C.V.  numbers.  It  should  be  noted  that  with 
crack  sizes  of  .020  in.  the  effects  of  F.S.  solution  errors  (e.g.  C.V.  values) 
are  less  than  for  larges  cracks  of  .100  Inch.  Mean  and  C.V.  values  tor  K.. 
were  obtained  from  laboratory  test  data,  'iha  material  yield  strength  of  IdO 
KSI  provided  an  upper  bound  for  calo-jlated  a (allowable  stress)  obtained  from 
the  Monte  Carlo  Fracture  Mechanlt.^  Method  (F7S.S.). 

Table  IV  provides  a partial  listing  of  reliability  for  the  ATP  system 
where  crack  sizes  are  ,100  and  .050  Inches  with  variations  In  C.V.  of  5% 
to  20X.  The  effects  of  F.E.  errors  ere  noted  as  In  Tables  III,  small 
variation  better  reliability,  large  variation  poorer  reliability.  The 
reliability  numbers  in  parenthesis  arc  results  from  application  of  the 
series-parallel  method  described  In  the  text. 

The  series-parallel  method  which  requires  failure  of  all  adjacent 
elements  in  order  to  have  structural  failure  provides  much  less  conservative 
estimate  of  reliability.  It  la  poeslble  that  an  upper  and  lower  bound  on 
reliability  of  this  structure  for  the  specified  crack  sizes  could  be  a 
series-parallel  system  end  the  woekest  link  approach  respectively.  With 
tension  end  compressive  stresees  exlatlng  In  this  structure  It  does  not 
seem  advisable  to  consider  structural  failure  In  terms  of  any  given 
element  failure.  It  also  unreasonable  to  assume  that  all  adjacent  elements 
must  fall  to  have  structural  failure  particularly  if  a bending  stress  exists 
In  the  structure. 

Although  the  Monte  Carlo  (F.M.)  Is  hypothetical  method  for  estimating 
structural  reliability,  It  does  provide  a desirable  alternative  to  the 
present  Fracture  Mechanics  approach  which  assumes  cracks  oriented  transverse 
to  maximum  principal  straea  resulting  In  an  unnecessarily  high  rejection 
rates.  The  ability  of  examining  at  least  qualitatively  the  reliability  of 
structures  as  related  the  ability  of  detecting  flaws  or  cracks  of  various 
magnitudes  can  provide  a guide  for  future  NDT  development  procedures.  If 
more  information  was  known  regarding  structural  flaw  distributions  the 
Monte  Carlo  method  could  provide  an  excellent  reliability  tool  certainly 
superior  to  the  present  lab  test  procedures  applied  to  brittle  materials. 

In  laboratory  testing  surface  flaws  are  often  removed  from  material  thereby 
preventing  an  accurate  representation  of  the  materials  strength  as  it  is 
related  to  the  structural  component.  ' 

In  application  of  the  Monte  Carlo  method.  Determination  of  proper 
number  of  simulation  In  Monte  Carlo  was  obtained  from  examlng  the  con- 
vergence rate  for  the  calculated  reliability  numbers.  Instead  relying  on 
some  elaborate  formulation  for  establishing  proper  number  of  simulations, 
a chart  similar  to  the  one  shovm  In  fig.  6 was  used.  All  functional 
parameters  were  increased  equally  in  number  In  order  to  examine  over  all 
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Table  III 


Fracture  Mechanics  Method 


Table  IV 


efftcts  of  the  elnuletion.  In  order  to  examine  the  acceptability  of  this 
method,  a comparison  was  made  for  R using  the  Warner  diagram  approach  where 
normal-normal  P.D.F.'s  were  calculated  from  a prescribed  mean  and  S.D.  for 
a tress-strength  values.  Results  show  excellent  agreement  with  Monte  Carlo 
simulation  method  using  the  convergence  rate  approach. 
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Where  f(s}  and  f(S)  are  PDF  representation  for  stress 
and  strength  values  respectively. 


Psk  - Probability  of  Survival  of  Element  K 


■ifi 


Appendix  C 


Series  - Parallel  Concept  (Tension-Compression) 


Assume  simply  four  element  structure 


M 

l^sal 

Ps,  ' 

1 ^S4 

Ps2  Ps3  f’S4 

Psi2  “ Psi  Ps2  “ Psi  ' Ps2 


Survival  Probability  ror  Structure 


P 


p 


For  large  structures  all  elements  considered  witli 
their  corresponding  adjacent  elements. 
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.\BSTRACT.  Methods  were  investigated  for  estimating  the  mechanical  behavior  of  various 
materials  when  subjected  to  weapon  setback  forces.  A mathematical  model  was  developed  for 
predicting  the  dynamic  behavior  of  the  materials  from  which  predictions  concerning  certain 
material-geometry  combinations  were  made.  Tests  were  carried  out  with  aluminum  and  steel 
specimens  and  the  resultant  data  were  compared  with  results  predicted  by  the  mathematical  model. 
The  mathematical  model  selected  was  a viscoelastic  model  involving  viscous,  elastic,  and,  in  one 
instance,  plastic  parameters. 

I.  INTRODUCTION.  The  design  of  chemical  munitions  Involves  several  factors,  among 
them  is  the  structural  behavior  of  the  munition  components  when  subjected  to  various  loads.  One 
important  factor  is  the  forces  imposed  by  the  launch  environment  which,  for  artillery  rounds,  is 
referred  to  as  setback.  At  present,  the  developmental  emphasis  is  on  system  testing  which  yields 
information  on  a specific  configuration;  and,  if  the  configuration  is  changed,  the  system  must  be 
retested.  In  a few  cases,  bench  testing  supplements  system  testing  and  although  the  information 
obtained  from  the  bench  tests  is  more  general  than  from  system  tests,  it  is  limited  to  the 
geometry-material  configurations  tested.  These  two  methods  have  resulted  in  increased  project 
costs  and,  at  times,  in  schedule  slippages. 

What  is  required  is  an  alternate  and  somewhat  more  general  method  for  analyzing  the  effects 
of  the  launch  environment  on  munition  components.  Therefore,  an  investigation  was  undertaken  to 
develop  such  a method  and,  thereby,  reduce  the  amount  of  munition  flight  testing  and/or  dynamic 
testing  required  with  its  associated  costs. 

II.  BACKGROUND.  Although  the  plastic  deformation  of  solids  at  high  strain  rates  was 
commented  on  as  early  as  1904  by  Hopkinson,*  it  was  not  until  1941  when  VonKarman*  and 
Taylor  each  independently  established  the  theory  of  plastic  propagation  in  metals  that  this 
phenomenon  was  seriously  studied.  Since  the  publication  of  this  theory,  a body  of  research  data  has 
grown.  In  general,  these  data  can  be  summarized  into  feur  general  statements  which  are:  (l)The 
displacement  mechanisms  for  metals  subjected  to  impact  loads  are  similar  to  those  for  statically 
applied  loads  but  they  have  different  distributions;3  (2)  there  exists  a time  delay  between  the 
application  of  impact  loads  and  subsequent  plastic  yielding;^  (3)  materials’  strength  increases  when 
subjected  to  impact  loads,  but  by  varying  relative  amounts  for  various  materials,  and  the  strain  rate 
affects  both  the  strength  and  ultimate  elongation  of  a material;^ and  (4)  previous  investigations 
showed  that  flow  or  viscous  parameters  are  important. 

III.  MODELING  EFFORT.  Based  on  the  information  obtained  during  the 
above-mentioned  literature  search,  a mathematical  model  has  been  developed  to  predict  the 
time-dependent  behavior  of  material.  This  model  assumes  that  materials  behave  in  an  elastic  manner 
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below  the  proportional  limit  (arbitrarily  aet  at  2%  elongation)  and  in  a viscoelastic  manner  above 
the  proportional  limit.  In  addition,  the  viscoelastic  element  must  have  a time  delay  or,  in  this  case,  a 
viscous  resistance  feature  which  is  known  as  a Voigt  element.  The  mechanical  analog  of  the  model  is 
given  in  figure  1 . 


Ee 


de  . 

+ E« 


Figure  I . Mechanical  Analog  of  Elutic-Vlicoelutic  Model 

The  response  equation  for  this  model  is  the  sum  of  the  two  equations  given  to  the  right  of 
figure  1.  The  time-dependent  strain  for  this  system  is  found  by  solving  this  equation  for  the  model 
when  excited  by  the  forces  due  to  the  launch  environment.  Once  the  strain  relationship  is  known, 
the  other  dynamic  values  can  be  determined.  The  dynamic  values  can,  in  turn,  be  substituted  into 
the  equations  describing  the  specific  geometry  of  Interest  which  will  yield  predicted  values.  This 
methodology  requires  two  types  of  data:  first,  an  estimate  of  the  forces  imposed  by  the  weapon 
system  which  can  be  obtained  from  acceleration/pressure  histories  of  the  weapons  system  and 
second  the  elastic  and  viscous  constants  of  the  material,  which  may  be  obtained  from  a statically 
obtained  true  stress-strain  diagram  by  assuming  that  the  material  behaves  elastically  below  the 
proportional  limit  and  in  a viscous  manner  above  it.  llie  graphical  representation  of  this  behavior  Is 
shown  in  figure  2. 

IV.  EXPERIMENTAL.  To  compare  the  model  predictions  with  experimental  data,  a 
series  of  dynamic  bench  tests  was  performed  Involving  two  geometries  (the  thin  flat  plate  as  shown 
in  figure  3 and  the  solid  pin  as  shown  in  figure  4)  and  four  materials  (AISl  1018  and  1020  low 
carbon  steel,  type  316  austenitic  stainless  steel,  and  6061-T6  aluminum  alloy).  The  stress-strain 
diagrams  for  all  four  materials  were  generated  to  provide  elastic  and  viscous  material  constants,  as 
shown  in  figure  S. 
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Fi|ur<  2.  Ideilized  True  Streu-Stratn  Dlagrtm 


Figure  3.  Flat-Plate  Conflguration 


The  bench'teit  machine  used  in  this  work  was  a drop  table  shock  machine  which  produces  a 
controlled  and  reproducible  acceleration  at  a constant  pulse  rate.  Tbe  acceleration  history  for  the 

bench'test  machine  used  is  a half  sine  wave  pulse  Ia(t)"  ap8in^~jt].  The  response  equation  for 

this  system  is; 


+ Ee-map8in^^^t 


where  the  material  response  characteristics  are  on  the  left  and  the  excitation  forces  are  on  the  right 
side  of  the  equal  sign.  The  time  dependent  strain  for  this  system  is: 


t? 


. (f)’ '(»’ 


which  was  obtained  by  solving  the  response  equation.  The  dynamic  modulus,  that  is,  the  stress 
divided  by  the  strain,  for  this  system  is: 


In  this  work,  the  burst  strength  of  a flat  plate  or  the  shear  strength  of  a solid  pin  was  to  be 
meuured;  consequently,  the  system  was  evaluated  at  the  peak  stress  ^t  ■ This  occurs  when  the 

trigometric  flinction  in  the  numerator  is  equal  to  one.  In  addition,  for  these  tests  the  pulse 
duration  (T)  was  In  the  millisecond  range  which  flirther  simplifies  the  modulus  to 
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The  stress  on  a flat  plate  can  then  be  found  by  solving  the  equation  of  motion  for  that 
geometry 

- 0 2 k3  n2  SIh. 

8t2  ” ax^ 


subject  to  theequation  of  continuity  v^  " which  is: 
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where  the  strew  wave  propagation  velocity,  Cg,  equals  (~)  ; radius  of  gyration,  K,  equals  I 

half  wave  length,  A,  equals  VgT;  and  A equals  ^1 -f  which  accounts  for  radial  and 

circumferential  displacement.  The  stress  on  a solid  pin  can  also  be  found  by  solving  the  equation  of 
motion  for  that  geometry,  which  is  analogous  to  that  for  a flat  plate. 
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V.  RESULTS.  The  reaults  of  the  dynamic  bench*test  effort  are  given  in  the  table.  This 
work  involved  flve  material-geometry  combinations  and  because  of  the  “go,  no«go’’  nature  of  the 
test  setup  only  upper  and  lower  strength  values  are  reported.  Along  with  the  test  values  are 
estimates  of  the  precision  of  the  shock  machine,  the  elastic*viscoeIaatic  model  predictions,  and,  for 
comparative  purposes,  strength  predictions  based  on  statically  obtained  material  properties  using 
standard  formulas. 


VI.  DISCUSSION.  In  comparing  the  elastic>viscoelastic  model  predictions  with  the 
bench-test  values  for  low  carbon  steel  and  aluminum  alloy,  it  is  seen  that  the  two-element  model 
does  adequately  predict  the  behavior  of  these  two  metals.  In  comparing  the  two  methods  of 
predicting  strength  values,  it  Is  seen  that  the  predictions  baaed  on  the  elastloMacoelastlc  model  are 
far  superior  to  those  made  using  standard  strength  values  and  formulas. 

For  the  austenitic  stainless  steel  type  316,  the  elastic-viscoelutic  model  predicted  values 
considerably  higher  than  the  upper  bench-test  value  for  the  solid-pin  geometry  (9,232  lb  versus 
2,SS2  lb).  The  most  obvious  explanation  for  this  discrepancy  is  that  the  model  as  proposed  does  not 
adequately  account  for  the  behavior  of  316  whereas  it  does  so  for  low  carbon  steel  and  the 
aluminum  alloy. 

Both  the  low  carbon  steel  and  6061-T6  aluminum  alloy  are  basically  pure  metals  doped  with 
small  percentages  of  other  elements  to  achieve  certain  desirable  mechanical  properties.  In  the  case 
of  the  aluminum  alloy,  the  presence  of  these  impurities  in  the  crystal  lattice  tends  to  produce 
barriers  to  metal  flow  which  can  be  modeled  as  a viscous  drag  element.  In  the  case  of  low  carbon 
steel,  the  presence  of  these  impurities  plus  a susceptibility  to  strain  hardening^  likewise  tend  to 
produce  barriers  to  metal  flow  which  can  be  modeled  as  a viscous  drag  element. 

On  the  other  hand,  austenitic  stainless  steel  type  316  is  a mbcture  of  iron,  chromium,  and 
nickel  where  the  chromium  and  nickel  atoms  substitute  for  iron  in  the  crystal  lattice.  Because  of  the 
high  nickel  content  (10%-]  4%),  this  steel  does  not  appreciably  strain  harden  and  is  sometimes 
referred  to  w free  spinning  steel.''  This  relative  lack  of  strain  hardening  has  been,  in  part,  explained 
by  an  “easy  glide”  mechanism  where  the  density  of  dislocations  rises  linearly  with  plastic  strain.*  In 
fact,  the  presence  of  impurities  would  be  the  only  serious  barrier  to  metal  flow  and  is  overshadowed 
by  the  plasticity  of  the  metal. 

Since  the  austenitic  stainless  steel  does  not  follow  the  elastic-viscoelastic  model  previously 
stated,  a new  and  more  general  model  which  accounts  for  the  behavior  of  this  materi^  must  be 
postulated.  Baaed  on  the  above  discussion,  a three-element  model  shown  in  flgure  6 is  proposed. 
This  model  involves  an  elastic  element  to  account  for  the  behavior  below  the  proportional  limit, 
followed  by  a Voigt  element  to  account  for  the  impurities,  followed  by  a friction  element  to 
account  for  the  “easy  glide”  phenomenon.  The  response  equation  for  this  system  is  the  sum  of  the 
three  individual  equations  given  in  flgure  6.  The  solution  for  the  thiee-element  model  is  analogous  to 
that  for  the  previously  discussed  elutic-viscoelastic  model. 
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Plgurt  6.  Mechinictl  Anilog  of  Etiitlc>VUcoelutlc-Plutic  Model 

In  analyzing  the  true  itreu-strain  diagram,  three  regions  must  be  considered  - elastic 
(0%-2%  strain),  viscoelastic  (2%-3.9%  strain)  or  tran^tional,  and  plastic  (3.9%-59%  strain)  - instead 
of  two  regions  as  in  the  previous  model.  This,  in  effect,  reduces  the  value  of  the  viscous  constant, 
thereby  reducing  the  predicted  shear  strength  from  9,232  to  2,833  lb.  In  comparing  the  three 
methods  of  predicting  strength  values  for  stainless  steel,  type  316,  it  is  seen  that  the  prediction 
based  on  the  three<«lement  model  is  superior  to  those  made  using  both  the  standard  formulas  and 
the  two^lement  model. 

VII.  SUMMARY.  The  mechanical  behavior  of  materials  subjected  to  impact  loads  can  be 
predicted  by  using  relatively  simple  mathematical  models  to  describe  their  behavior,  two  of  which 
have  been  discussed.  The  selection  of  model  depends  on  the  general  form  of  the  true  stress-strain 
diagram  of  the  candidate  material.  The  model  parameters  can  also  be  obtained  from  a true 
stre8^strain  diagram. 
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ABSTRACT.  The  use  of  Stewartson’s  instability  theory,  in  the  design  of  llquid*fllled 
projectiles,  la  a valuable  tool  for  the  munitions  designer.  A summary  of  Stewartson’s  theory, 
expanded  to  include  viscous  and  liquid  spin-up  effects,  is  presented.  Design  procedures  for  applying 
the  theory  with  an  example  are  shown.  Three  examples,  where  the  theory  has  been  successfully 
applied,  are  also  given. 

1.  INTRODUCTION.  Early  efforts  in  the  design  of  liquid-filled  projectiles  gave  engineers 
problems  primarily  because  there  was  not  a clear  understanding  of  the  liquid  instability  mechanism. 
When  early  projectile  designs  became  unstable  during  flight,  it  was  postulated  that  the  failure  of 
liquid  to  attain  fliU  spin  (l.e.,  rigid  body  rotation)  prior  to  shot  exit  was  the  chief  cause  for 
projectile  instability.  Longitudinal  baffles  were  designed  which  effectively  compartmentlzed  the 
projectile  cavity.  The  baffles  spun  the  liquid  up  as  the  projectile  casing  spun  up  in  the  tube; 
however,  this  was  not  sufficient  to  ensure  that  the  projectile  would  be  dynamically  stable.  Two 
problems  encountered  were:  (1)  Even  though  the  liquid  was  completely  spun  up  prior  to  shot  exit, 
the  baffles  in  the  projectile  could  have  been  positioned  so  that  asymmetries  existed  between  the 
center  axis  of  the  projectile  and  the  center  of  the  baffles  and  because  of  this  asymmetry,  which  could 
cause  the  projectile  to  be  unstable,  the  designer  could  not  predict  whether  or  not  two  similar 
projectiles  would  be  balllstically  stable;  (2)  difficulties  were  also  encountered  in  fabricating  the 
transverse  baffles  due  to  projectile  internal  tolerances. 

\ 

In  an  effort  to  observe  the  effect  of  liquids  on  the  stability  of  projectiles,  experiments  were 
conducted  In  which  lightweight  projectiles  were  fired  at  low  velocities.  Low  launch  velocities 
enabled  visual  observations  of  projectile  flight  behavior.  The  problem  encountered  with  this 
technique  was  that,  unless  flight  behavior  was  dramatic,  it  could  not  determine  if  the  wind  and/or 
liquid  caused  the  observed  behavior. 

The  only  theory  explaining  the  effect  of  liquid  in  a spinning  body  was  Greenhill’s  work 
related  to  spherical  cavities.  The  theory  showed  that,  if  a spinning  spherical  top  containing  liquid 
was  disturbed,  the  liquid  was  disturbed  to  a lessor  degree.  The  liquid  disturbance  created  natural 
oscillations  which  were  characterized  by  discrete  eigenfrequencles.  The  important  aspect  of  the 
theory  was  that,  if  the  natural  frequency  of  the  liquid  coincided  with  a natural  frequency  of  the 
top,  the  top  could  exhibit  unstable  motions.  The  coincidence  of  the  two  frequencies  created  a 
hydrodynamic  couple,  known  as  resonance.  It  turns  out  that,  for  a spinning  spherical  liquid-filled 
cavity,  only  one  characteristic  frequency  causes  resonance. 

Stewartson’s  theory*  analyzed  the  liquid  effect  on  a spinning  top.  Tlie  interior  geometry  of 
the  spinning  top  was  a right  circular  cylinder.  Stewartson  found  a double  infinite  number  of  modes 
of  oscillation  which  would  affect  top  stability  unlike  the  sphere  with  only  one  mode.  The  modes  are 
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characterized  as  radial  (n)  or  axial  (j)  halfSvaves.  Resonance  between  the  nutational  frequency  of  the 
projectile  and  a natural  frequency  of  the  liquid  causes  dynamic  instability.  However,  an  important 
outcome  of  Stewartson’s  theory  was  that  the  designer  could  control  resonance  by  varying  the  fill 
ratio  and  cavity  fineness  ratio  (internal  length/diameter  ratio).  Stewartson’s  theory  has  been  a 
primary  design  tool  in  the  design  and  development  of  liquid'fllled  munitions.  The  purpose  of  this 
paper  is  to  show  how  the  theory  has  been  utilized  for  the  design  of  liquid-filled  projectiles.  No 
mathematical  derivation  of  Stewartson’s  theory  will  be  provided  as  it  is  available  in  literature. 

II.  DYNAMICS  OF  LIQUID-FILLED  PROJECTILES.  The  paper  will  not  discuss  in  detail 
the  dynamics  of  liquid-filled  projectiles  but  will  just  mention  a few  key  principles.  The  spinning 
liquid  exerts  pressure  forces  on  the  inner  wall  of  the  projectile  and  also  is  excited  to  a lesser  degree 
when  the  projectile  is  subjected  to  an  external  disturbance.  These  two  points  show  that  there  exists 
a complex  interaction  between  the  spinning  projectile  and  internal  liquid.  The  Navie^Stokea 
equations  describe  the  perturbed  fluid  motion  ^thin  the  spinning  projectile.  The  following 
boundsry  conditions  are  used  with  the  Navier-Stokes  equation:  (1)  Pressure  is  a constant  on  a free 
surface,  (2)  tangential  components  of  the  flow  velocity  and  the  velocity  of  the  wall  are  equal,  and 
(3)  the  normal  components  of  the  flow  velocity  and  surface  velocity  are  equal.  A complete 
mathematical  derivation  can  be  found  in  literature. 

III.  STEWARTSON’S  THEORY.  Wlien  Stewartson’s  theory  was  applied  to  the  design  of 
liquid-filled  projectiles,  the  results  were  twofold:  (l)The  theory  provided  the  knowledge  of  the 
instability  mechanism-resonance  between  the  liquid  eigenfrequenoy  and  projectile  nutational 
frequency  and  (2)  the  theory  provided  designers  a quantitative  means  for  designing  liquid-filled 
projectiles.  The  theory  does  not  definitely  predict  the  design  will  be  stable  but  it  does  definitely 
state  if  a design  is  unstable. 

Stewartion  developed  his  theory  using  the  following  assumptions;^ 

1.  The  internal  cavity  is  a rigut  circular  cylinder  this  limits  the  shape  where  the  theory 
is  applicable.  However,  in  the  case  of  binary  projectiles,  currently  in  development,  right  circular 
cylinders  are  used  to  contain  the  liquid. 

2.  The  overturning  moment  is  the  only  significant  aerodynamic  force.  By  neglecting  the 
other  aerodynamic  moments,  we  can  concentrate  on  the  one  force  which  will  affect  projectile 
stability  with  a liquid-filled  payload. 

3.  The  projectile  flies  with  constant  velocity  and  spin.  In  actuality,  projectile  spin 
decresses  throughout  flight  where  projectile  velocity  will  decrease  until  apogee  and  then  increase 
until  impact.  This  assumption  is  chosen  mainly  for  convenience  (steady  state). 

4.  The  liquid  is  in  rigid  body  rotation  at  shot  exit  with  its  spin  identical  to  that  of  the 
projectile.  This  neglects  liquid  spin-up  which  does  vary  depending  on  liquid  and  launch  conditions. 
However,  it  is  a good  assumption  for  flnt  approximations. 


5.  The  centrifugal  force  of  the  liquid  is  so  much  larger  than  the  gravitational  or  drag 
forces  - the  other  forces  can  be  neglected  in  the  analysis. 

6.  The  mass  of  liquid  is  small  compared  to  the  mass  of  the  total  projectile  (this  was  taken 
for  convenience).  However,  in  actuality,  it  is  a good  assumption  since  most  binary  liquid  payloads 
sre  approximately  10%  of  total  projectile  weight. 
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7.  The  liquid  is  incompressible  and  inviscid.  Incompressibility  is  a good  assumption  for 
the  liquids  used;  however,  viscosity  plays  an  Important  role  in  liquid  spin-up. 

8.  External  shell  disturbances  are  restricted  to  small  yaw  amplitudes  and  the  liquid 
experiences  only  small  perturbations  due  to  this  external  disturbance  - the  projectile  can 
experience  large  yaw  amplitudes.  However,  the  assumption  is  acceptable  to  initial  design  procedures 
for  Uquid-flUed  projectiles. 

Stewartson’s  theory  provides  the  designer  with  two  important  facts: 

1.  The  projectile  yaw  will  grow  without  limit  under  certain  conditions.  Figure  1 depicts 
normal  projectile  yaw,  whereas  flgure  2 shows  yaw  growth  without  limit. 

2.  The  liquid  conforms  to  cavity  motions  through  excitation  of  small  amplitude 
oscillations  superimposed  on  the  rigid  body  rotation. 

Stewartson's  instability  criterion  for  minimizing  resonance  is:^ 


where  ••  nutational  frequency  of  projectile 
Tq  ■ natural  frequency  of  liquid 
S ■■  Stewartson's  parameter 

- p[2Rl  V 
1^0  c/a 

where  p ■ density  of  liquid 

2R  ■ pole  value,  Stewartson  table 
a ■ cavity  radius 
c ■>  cavity  length 
■ axial  moment  of  inertia 
o"  (1  - 1/Sg)^ 

Sg  ■ gyroscopic  stability 
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IV.  VISCOUS  EFFECTS.*  Stewartson  assumed  the  liquid  to  be  inviscid.  However,  the 
viscosity  of  the  liquid  changes  Stewartson’s  overall  theoiy  in  the  following  ways;  (1)  The  viscosity 
of  the  liquid  will  shift  any  liquid  eigenfrequency  slightly,  and  (2)  the  instability  band  for  resonance 
is  broadened.  The  inviscid  eigenfrequency  is 


IDlV 

2 L ^ 

When  viscous  effects  are  inciuded,  the  eigenfrequency  becomes 

2 L 4 olJ 


»bv"»'0+  ac/,(2j  + i)‘  a(;j  + l) 

The  difference  of  the  viscous  and  invisdd  liquid  eigenfrequency,  ArQ»  is  composed  of  a real  and  an 
imaginary  part  [tqy  -Tq  ■ Atq  ■ e - 15] , The  real  part  is  added  to  the  liquid  eigenfrequency  to  shift 
It  slightly  where  the  imaginary  part  broadens  the  resonance  band.  Stewartson’s  Instability  criterion 
is  modified  to 


5b  5c  Btq  2b^  5a 
a c _ 5b2/a2  a 


when  viscosity  is  taken  into  account. 


■i 


V.  SPIN-UP.  One  of  Stewartson’s  assumptions  was  that  the  liquid  is  at  rigid  body  rotation 
at  shot  exit.  This  assumption  has  been  experimentally  shown  (using  on-board  telemetry  in 
projectiles)  to  be  iitvalid.  The  liquid  spin-up  - deflned  as  the  process  where  the  liquid  acquires 
angular  momentum  - varies  depending  on  liquid  viscosity  and  projectile  launch  conditions.  As  the 
liquid  spins  up,  the  projectile  casing  spins  down  until  liquid  rigid  body  rotation  is  attained.  The 
liquid  close  to  the  wall  will  spin  up  due  to  the  frictional  force  between  the  liquid  and  the  projectile 
casing.  A second  spin-up  phenomenon  exists  which  is  deflned  as  Eckman  layers.  An  Eckman  layer  is 
formed  when  the  canister  ends  act  as  centrifugal  fans  sucking  the  liquid  from  the  nonrotating  center 
core  and  throwing  the  liquid  outward.  The  question  is  how  to  Include  liquid  spin-up  in  the  analysis 
of  a liquid-fllled  projectile.  Wedemeyer  has  shown  that  eigenfrequencies  in  a partially  spinning  fluid 
are  the  same  as  an  equivalent  cylinder  having  the  same  angular  momentum.  Knowing  this,  the 
designer  can  determine  the  stability  of  the  projectile  as  liquid  acquires  angular  momentum.  The 
designer  can  evaluate  spln-up  at  various  stages;  i.e.,  10%  spin-up,  20%  spin-up. 

The  curves  in  figures  3 and  4 depict  liquid  spin-up. 
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VI.  APPLICATION  OF  STEWARTSON*S  THEORY.  The  designer  must  apply 
Stewartson’s  theory  modified  for  viscous  effects  to  the  steady  state  condition  (no  liquid  spin'-up) 
and  then  examine  projectile  stability  during  liquid  spin-up.  The  following  design  steps  may  be  used 
as  a guide. 


1.  First  a liquid  fill  must  be  chosen.  The  internal  cavity  dimensions  will  be  designed 
dependent  on  the  liquid  chemistry  and  the  fact  that  the  liquid-fllled  projectile  should  weigh  the 
same  as  conventional  munitions  of  the  same  external  shape, 

2.  The  physical  characteristics  of  the  proposed  design  (moments  of  inertia,  weight)  are 
then  used  to  calculate  gyroscopic  stability.  Gyroscopic  stability  predicts  projectile  stability  at  shot 
exit. 

3.  The  viscosity  of  the  liquid  should  be  checked  to  see  if  it  is  sufficient  to  produce  viscous 
effects. 

4.  Determine  Stewartson’s  instability  criteria. 

5.  If  the  projectile  is  stable  in  steady  state,  the  stability  during  liquid  spin-up  should  then 
be  examined. 

The  following  example  will  illustrate  the  first  four  steps.  The  Fifth  step  is  lengthy  due  to  the 
number  of  iterations  needed  to  determine  stability  during  liquid  spin-up. 

lSS»mm  Projectile 

Step  1 : Determine  the  initial  physical  characteristics. 

Ix  » 19.525  slug-ft2 
ly  » 198.224  slug-ft2 
m <■  3.20  slug 
n ■ 20  cal/turn 
pa  » 0.002378  slug/ft^ 
d -0.51  ft 
a - 0.17  ft 
c/a  - 4.65 
CMa“5.96 
Mf  ■ 0.236  slug 

The  above  constants  are  projectile  physical  characteristics  calculated  from  step  1 or  Initial 
launch  conditions;  i.e.,  air  density,  bore  rifling,  and  static  moment  coefficient. 
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Step  3 : Determine  vitcoui  effects. 

V ■ launch  velocity  ■ 1027  ft/iec 

2ff 

0 “ “T-  V ■ 632.6  radiani/iec 
nd 

a ■ S.2  cm 
»»  - 1.1  X 10*  itcke 
Re  - 1.SSX  10^ 

• ■ 0.0003 
j - 0.0007 

I ft  S are  sufficiently  small  that  effects  due  to  viscosity  may  be  neglected. 

Step  4!  Calculate  Stewartson  instability  at  steady  state. 

Cavity  is  90%  ftill  or  b^/a*  - 0.02 
c/a  ■ 4.65 

Construct  the  following  table: 


J 

4.65/2j  + 1 

^0  (nJ) 

2R 

0 

4.65 

— 

- 

1 

1.55 

0.35(1,  I) 

1.359 

2 

0.93 

0.45  (2,  2) 

0.343 

3 

0.664 

0.24  (2, 3) 

0.119 

Stewartson’s  table  (b*/a*  - 0.02)  is  entered  knowing  columns  1 and  2 above,  where  radial 
modes  n - 1,  2,  3 are  checked  to  And  tq  (column  3)  and  2R  (column  4). 

Stewartson's  parameter  is  calculated 

S-0.38X  10-*(2R)* 

for  n ■ 2,  j ■ 3 

S - 5.38  X 10"*  y/S"  7.33  X 10"* 

.i<2ZId<, 

-1<9.96<  1 


The  instability  criteria  show  the  projectile  should  be  stable  at  steady  state. 


VII.  PROJECTILES  DESIGNED  USING  STEWARTSON’S  THEORY.  Designers  must  keep 
in  mind  that  the  chemical-fllled  round  is  normally  required  to  be  ballistically  similar  to  a 
conventional  round  in  the  same  family.  This  allows  the  use  of  conventional  rounds  as  spotter  rounds 
and  also  allows  the  use  of  the  same  ilring  tables  with  minor  corrections.  Projectiles  are  ballistically 
similar  when  the  following  are  nominally  the  same:  (1)  external  shape.  (2)  drag,  (3)  weight,  and 
(4)  internal  ballistic  characteristics.  However,  differences  in  physical  characteristics  may  exist, 
provided  that  the  effect  of  these  differences  on  the  means  of  the  burst  point  distribution  are 
predictable  and  can  each  be  compensated  for  by  the  application  of  constant  corrections  for  each 
charge. 


A.  M687  Projectile.  The  flrat  binary  round  to  apply  Stewartson’s  theory  was  the  M667 
projectile.  The  advanced  developmert  round  (see  figure  S)  was  designed  using  a computer  model  of 
Stewartson’s  theory.  Initial  dispersion  differences  for  Impact  points  were  attributed  to 
miscalculation  of  range  or  the  use  of  a faulty  propellant  or  both.  At  the  same  time,  failure  of  the 
burst  disks  to  shear  (which  allows  mixing  of  the  liquids)  and  also  changing  the  internal  cavity 
from  two  to  one  right  circular  cylinder  masked  an  instability  problem.  Increased  Instrumentation 
(on-board  telemetry  packages  and  high-speed  tracking  cameras)  showed  that  the  M687  round  had 
stability  problems.  The  M687  projectile  used  the  same  aerodynamic  shape  as  the  M483  improved 
conventional  munition,  a stable  design.  Therefore,  the  instability  was  attributed  to  the  liquid 
payload.  Analyses  of  the  computer  model  revealed  that  liquid  spin-up  was  neglected.  The  model 
predicted  steady  state  behavior.  The  model  was  revised  to  incorporate  the  liquid  spin-up 
phenomenon.  The  Incorporation  of  the  liquid  spin-up  resulted  in  the  shortening  of  the  internal 
cavity  by  3.8  cm.  Subsequent  field  trials  showed  the  design  to  be  stable  under  the  severest  launch 
conditions.  Figure  6 shows  the  final  design. 

B.  XM736  8-Inch  Projectile.  The  XM736  8-lnch  projectile  is  currently  in  the  development 
cycle.  It  has  the  same  aeroballistic  conflguration  as  the  M509  improved  conventional  munition.  The 
computer  model  of  Stewartson's  theory,  including  viscous  corrections  and  liquid  spin-up,  was  used 
to  arrive  at  the  present  design  (figure  7).  The  use  of  Stewartson’s  theory  minimized  the  possibility 
of  a liquid  stability  problem.  The  XM736  projectile  has  a very  stable  aerodynamic  shape;  that  fact, 
plus  the  adaptation  of  Stewartson’s  theory,  produces  a stable  design  even  under  the  severest  launch 
conditions. 


C.  155-mm  IVA  Projectile.  The  ISS-mm  IVA  projectile  is  currently  in  exploratory 
development.  Stewartson’s  theory  has  been  adapted  for  a number  of  liquid  nils.  All  the  designs  have 
been  successfully  flight  tested.  A typical  configuration  is  shown  in  figure  8. 

Even  though  Stewartson’s  theory  can  only  predict  unstable  rounds,  the  use  of  the  theory  has 
minimized  costly  testing  by  eliminating  poor  designs  on  the  drawing  board  prior  to  fabrication. 
Tremendous  success  has  been  achieved  in  the  design  of  liquid-filled  projectiles  by  adapting 
Stewartson’s  theory.  Currently,  at  Chemical  Systems  Laboratory,  an  independent  laboratory 
in-house  research  project  is  in  progress  to  expand  Stewartson’s  theory  for  an  internal  step  design. 
Stewartson’s  theory  assumes  a constant  inside  diameter.  Current  projectile  designs  may  require  a 
step  design  - two  different  Inside  diameters.  The  current  approach  for  design  of  such  projectiles  is 
to  assume  a mean  diameter.  So  far  this  has  been  successful.  However,  it  is  anticipated  that  a 
quantitative  method  will  result  from  this  research  study. 
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VlII.  CONCLUSION.  Stewartion'i  initabll<ty/stabillty  theory  modiHed  to  include  viacoui 
effecti  and  liquid  spin-up  provides  the  designer  with  a useful  tool  in  liquid-Hlled  projectile  design. 
The  use  of  Stewartson’s  theory  in  the  design  of  binary  chemical  rounds  has  been  very  successful. 
The  theory  has  reduced  costly  testing  by  computer  modeling  prior  to  fabrication  and  subsequent 
flight  tests. 
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TOTAL  TIME  ON  TEST  PLOTS 
Richard  E.  Barlow  and  Bernard  Davis 
University  of  California,  Berkeley 
Berkeley,  California 

1.  INTRODUCTION 

la  this  paper  wa  praaaut  a particular  graphical  technique  which  le 
very  uaaful  la  analyalag  failure  and  aurvlval  data.  The  central  concept 
In  reliability  theory  la  that  of  the  failure  rata.  Thla  la  a feature  of 
the  life  distribution.  In  fact.  If  the  life  distribution  la  eontlnuoua, 
Vhlch  la  very  often  the  ease,  the  failure  rate  uniquely  deteralnea  the 
life  distribution.  Failure  rata  and  aging  are  two  very  cloaely  related 
concepts.  For  instance.  If  the  unit  which  la  being  tested  or  la  In 
ssrvlee  does  not  age  with  tlae,  that  Is,  Its  residual  age  is  independent 
of  Its  present  age,  we  sight  eay  Its  life  distribution  has  constant 
failure  rate.  This  Is  the  well-known  eharacteristie  property  of  the 
exponential  distribution  l.e. 

F(x)  ■ 1 - exp  {-x/e}  e > 0 

X > 0 


Then, 


Prob  IX>x  + y 1 X>  x] 


Prob 


IX  > X 
Prob  [X  > 


■ exp  {-<x  + y)/e}/exp  {-x/e> 

- exp  {-y/e) 

- Prob  {X  > y)  . 

This  research  was  supported  by  the  Air  Force  Office  of  Scientific  Research 
(AFSC),  USAF,  under  Orant  AFOSR-77-3179  with  tlie  University  of  California. 
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It  can  be  shovm  quite  easily  that  the  only  contlnuoua  life  distribu- 
tion with  the  non-aging  property 

Prob  [X  > X + y | X > x]  - P[X  > y] 

Is  the  exponential  [3]. 

This  Is  an  ideal  life  distribution  where  the  units  of  every  age  have 
the  same  residual  life  diatrlbutlon  as  new  units • 

We  have  used  the  term  "failure  rate"  earlier  and  we  shall  give  a 
precise  definition  of  the  term  below. 

The  conditional  probability  that  a unit  of  age  t will  fall  in  the 
next  interval  of  length  x is 

p(x/t)  - t r 

P(t) 

where  f(t)  ■ 1 - P(t)  is  the  survival  function. 

The  failure  race  r(t)  at  time  t is  defined  to  be 


r(t)  -ll„i£^.±-£lr..lM 

3f*0  * P(t) 

. IM 

F(t) 


where  f(t)  la  the  density  function  of  F . Note  that  f(t)  will  exist 
if  F la  an  absolutely  continuous  distribution.  Moat  of  the  life  distribu- 
tions of  interest  are  absolutely  continuous,  so,  unless  we  specify  otherwise, 
we  shall  assume  that  the  density  function  exists  for  all  the  distrlbutione 
under  consideration. 

If  we  Integrate  the  failure  rate  function  between  0 and  x 
we  get 
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So  w«  that  tha  aurvlval  function  F • and  hanca  tha  dlatrlbutlon 
function  F , la  uniquely  datarmlnad  by  tha  failure  rata  function  r(t). 
For  tha  axponantlal  dlatrlbutlon, 

r(t)  • 


- 1/0 


lia.  tha  axponantlal  dlatrlbutlon  haa  oonatant  failure  vata. 

Other  claaaaa  of  life  dlatrlbutiona  of  Intereat  are  tha  Inovaaaing 
Failune  Rata  (IFR)  dlatrlbutiona  and  tha  Daoraaaing  Failura  Rata  (DFR) 
dlatrlbutiona.  Aa  the  name  auggeata,  a life  dlatrlbutlon  F la  IFR  If 
r(x)  la  increaalng  and  altnllarly,  F la  DFR  If  r(x)  la  decreaalng. 
Uaually,  itama  which  during  their  llfe-apan  are  aubjected  to  wear  tend  to 
have  life  dlatrlbutiona  which  are  IFR,  The  cloaeat  to  a conatant  failure 
rata  la  exhibited  by  life  dlatrlbutiona  of  electronic  componanta  which 
degrade  vary  alowly  and  than  only  at  the  atomic  level. 

The  Interpretation  of  IFR  la  that  aa  the  item  on  teet  geta  older,  the 
dlatrlbutlon  of  the  realdual  life  tenda  to  got  cloaer  to  eero  In  aome  aenae, 


f 
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In  terms  of  probabilities, 

Prob  [X  > X + y | X > x] 

decreaiM  In  x for  all  y > 0 , assuming,  of  course,  Prob  [X  > x]  > 0 
A similar  explanation  applies  to  DPR  distributions.  For  a rigorous 
traatmant  of  the  above  aaa  [8]. 


I I.  TOTAL  TIME  ON  TEST 


This  classification  of  life  diatrlbutlons  into  IFR,  DFR  and  constant 
failure  rate  distributions  la  of  particular  interest  in  reliability 
engineering  and  maintenance  planning.  For  example,  in  formulating  replace- 
ment policies,  If  It  la  known  that  the  life  distribution  Is  either 
exponential  or  DFR,  then  clearly  the  optimal  policy  Is  to  replace  only 
upon  failure. 

Let  us  now  suppose  that  we  have  a random  sample 

Xl'X2*  •••»  \ 

from  a life  distribution  F . 

Let  < X^2)  < •••  < 

be  the  ordered  observations.  Then, 

"«(!)’  ■ *(1) 

Is  defined  to  be  the  total  time  on  teat  till  the  1th  failure. 

In  general,  If  we  denote  by  n(u)  the  number  of  items  on  test  at 
time  u , 


T(x) 


n(u)  du 


Is  the  total  time  on  test  till  x . 
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The  total  time  on  test  we  have  defined  above  la  not  scale-independent 
so  usually  It  Is  scaled  by  dividing  by  T(X^^j)  , 

Define 


Then  H (1/n)  is  called  the  scaled  total  time  on  test  till  the  1th 
n 

failure  and  the  plot  of  (l/n,H^(l/a))  where  adjacent  points  are  joined 
by  straight  lines  Is  known  as  the  empirical  scaled  total  time  on  test. 

We  define  H^(0)  ■ 0 so  that  the  plot  lies  In  the  unit  square  and  Is 
0 at  0 and  1 at  1 . 

If  the  underlying  life  distribution  were  really  exponential  It  can  be 
quite  easily  shown  (see  [2])  that  •••>  jointly 

distributed  like  the  order  statlstica  from  a sample  of  size  n - 1 from 
a Uniform  [0|1]  distribution.  Hence,  we  might  expect 
close  to  1/n  so  that  the  plot  would  be  quite  close  to  the  diagonal. 
Figure  1 shows  an  empirical  scaled  total  time  on  test  computed  from  a 
simulated  exponential  distribution  with  n ■ 20  . 

The  theoretical  basis  for  considering  this  plot  as  an  exploratory 
tool  for  determining  whether  the  underlying  distribution  Is  IFR,  DFR  or 
exponential  Is  as  follows: 

Lac  F (n)  be  the  empirical  distribution  function  determined  by 
n 

the  sample. 
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I 

i 

1 

I 

! 


l.e. 


(“ 

If 

X < 

a/n 

If 

X(i)  < X < X 

( 1 

If 

*(n)  - * • 

(i+1) 


Th«n  It  can  be  shown  quite  eaally  that 

*<i) 


i T(X,„)  - / (1  - »„(u))  dn 


Now  define  a transform  of  the  actual  distribution  function  as 
follows 


F"^(t) 

H“^<t)  - r (1  - F(u))  du  0 < t £ 1 . 

0 

The  slope  of  at  t ■ F(x)  can  be  shown  to  be 

Suppose  that  F has  finite  expectation,  then  the  plot  of 

(t,  — i Is  known  as  the  scaled  total  time  on  test  plot  of  F . 

Hp  (1) 

Figure  2 shows  scaled  total  time  on  test  plots  of  selected  Welbull 
distributions.  The  significance  of  this  plot  Is  that  the  total  time  on 
test  plot  of  an  exponential  distribution  Is  the  diagonal,  that  of  an  IFR 
distribution  is  a concave  curve  and  that  of  a DFR  distribution  is  a convex 
curve.  Note  that  the  Welbull  family  has  both  IFR  and  DFR  members 
depending  on  the  shape  parameter. 
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It  has  been  proved  (see  [2])  that  aa  the  sample  size  Increases  the 
empirical  plot  converges  to  the  total  time  on  test  plot  o£  the  actual 
distribution  oo  that  the  empirical  ecaled  total  time  on  test  Is  a very 
valid  graphical  tool  to  study  the  failure  rate  with. 

Figure  3 Is  based  on  failures  of  engines  of  a certain  model  tractor. 

Data  on  22  tractors  were  available  and  we  had  the  times  of  failures  of  the 
tractors  due  to  engine  failure.  When  the  original  engine  failed  it  was 
replaced  by  a reconditioned  one  and  when  It  failed  It  was  replaced  by  yet 
another  reconditioned  engine.  An  analysis  of  the  life  times  of  the  engines 
showed  that  the  original  engines  had  life  times  which  were  quite  IFR  but 
the  reconditioned  engines  were  not  quite  as  IFR  (see  [4]).  For  a related 
paper  which  contains  several  kinds  of  data  see  [1]. 

When  the  underlying  life  distribution  is  exponential  we  saw  that  the 
empirical  scaled  total  time  on  test  tends  to  lie  close  to  the  diagonal. 

This  fact  can  be  exploited  to  formulate  a very  simple  test  for  axponantlal- 
Ity.  If  the  plot  were  to  follow  the  diagonal  we  might  expect  It  to  cross 
the  diagonal  a number  of  times.  Then  the  number  of  crossings  of  the  diagonal 
by  the  plot  could  be  used  as  a test  statistic.  This  was  proposed  by 
Barlow  and  Campo  [2]  and  they  computed  the  sampling  distribution  of  the 
number  of  crossings  when  n ■ 20  using  simulation.  The  exact  distribution 
of  the  number  of  crossings  has  been  computed  by  Bergman  [5]. 
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III.  APPLICATIONS 


One  application  of  the  total  time  on  test  plot  Is  a method  for 
determining  graphically  optimum  replacement  policies.  We  will  use  the 
tractor  engine  data  for  an  example.  Suppose  Is  the  coat  of  a 

tractor  falling  In  service  and  c^  Is  the  cost  of  pulling  out  a tractor 
and  replacing  It  during  a planned  maintenance  period.  We  will  assume  that 
> C2  . Let  F denote  the  underlying  life  distribution  and  let  R be 
the  planned  operating  time  between  overhauls  of  the  engine.  Then  the 
expected  long  run  cost  of  using  the  maintenance  policy  R is 

c.F(R)  + c,(l  - P(R)) 

C(R)  - 

f(l  - F(u))  du 


since  F Is  unknown  we  replace  It  with  F , the  empirical  dlstrlbu- 

n 

tion  function.  Then,  given  c^  and  C2  we  want  to  determine  the  optimal 
R by  minimizing 


C(R) 


CjF^(R)  + Cjd  - Fp(R)) 
_ 

J(1  - F^(u)>  du 
0 


Usually  this  Is  done  numerically  but  we  can  use  the  total  time  on 
test  plot  to  do  It  much  more  easily. 

Recall  that  the  total  time  on  test  till  the  1th  failure  was 
defined  to  be 


*(i) 

f n(u)  du 
0 
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'(1) 

- n y (1  - F^(u))  du  . 

0 


So  that. 


'(1) 


(s)-  «*(!)>  ■ / a - '„(>■» 


Now,  nlnlialzlng  C(R)  la  equivalent  to  maximizing 


g/p)  ♦ T(R) 


C(R)  + C2<^  " 


(1/p)  ■ T(R) 


Ta,_x) 


isH  . 


T(R) 


/T(X.  v) 


(n)^  . 


n(o,  - c,) 


+ F^(r) 


“1  '■2'  - ^2  " 


Clearly,  maximizing  the  above  la  equivalent  to  maximizing 


T(R)/T(X^^^^) 


'1  ‘•2 


(*) 


To  get  the  R which  maximizes  the  above,  draw  the  tangent  from 


" ®2 


to  the  empirical  scaled  total  time  on  test  plot. 
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Let  8 denote  the  abscissa  of  the  point  of  tangency.  Then  is 

the  desired  value  of  R . From  the  graph  (Figure  4)  It  Is  seen  that  the 
■lope  of  the  tangent  la  the  maximum  value  of  (*) . 

This  procedure  can  be  implemented  on  an  interactive  computer  graphics 
system  to  determine  the  sensitivity  of  R to  different  values  of  the  costs 
c^  and  &2  • course,  the  major  drawback  of  this  method  is  that 
if  the  empirical  total  time  on  test  plot  is  based  on  a very  unrepresentative 
■ample  then  the  resulting  value  of  R may  be  worthless.  On  the  other  hand, 
if  F were  known  we  would  use  the  transform  of  the  actual  distribution 
and  the  same  method  can  be  applied  to  the  plot  of  the  transform  to  compute 
R . In  Figure  4 we  have  taken  c^  ■ 3c2  and  the  optimum  replacement 
policy  is  to  overhaul  the  engines  after  4815  operating  hours.  For  related 
work  see  [4],  [6]. 


3: 


Original 


IV.  THE  BIVARIATE  CASE 


One  area  of  reliability  theory  where  not  much  work  has  been  done  is 
in  graphical  analysis  of  multivariate  data.  The  problem  has  been  to 
represent  higher  dimensional  data  or  statistics  on  paper  or  any  2-dimenaional 
medium.  Some  initial  attempts  have  been  made  to  tackle  the  problem  in 
the  bivariate  case.  Ue  present  below  a technique  for  extending  the  total 
time  on  test  concept  to  bivariate  data. 

In  the  univariate  caea  the  life  distribution  was  completely  determined 
by  the  failure  rate  function  r(x)  and  the  total  time  on  teat  plot  was 
used  to  determine  the  behaviour  of  r(x)  . In  the  bivariate  case,  in 
addition  to  aging  we  have  the  added  complication  of  dependence.  So  instead 
of  having  just  the  failure  rate  we  have  a hasard  gradient  which  is  written 
as 

•fCx.y)  - (rj^(x,y),r2(j{,y)) 

As  in  the  univariate  case,  if  the  underlying  life  distribution 
F(xty)  on  [O.*)  x [0,*)  is  absolutely  contlnuoua,  then  r(x,y)  determines 
F(x,y)  uniquely.  In  fact  it  can  be  shown  [9]  that 

( (*.y) 

?(x,y)  « exp  |-  J y(f)dg 

where  P(x,y)  ■ Prob  {X  > x ,1  > y}  and  the  line  Integral  is 
path-independent . 

The  Interpretation  of  the  hazard  gradient  is  as  follows: 
r^(x, y)  - Conditional  failure  rate  of 
X given  Y > y . 
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laiiusif  jMia  uee^ 


~ Conditional  failure  rate  of 


Y given  X > X . 

One  of  the  properties  of  the  hasard  gradient  Is  that  the  behaviour 
of  rj^Cx.y)  as  y varies  and  the  behaviour  of  rjCx.y)  as  x varies 

describe  the  nature  of  the  dependence  between  X and  Y . For  Instance 

If  r^^Cx^y)  la  decreasing  In  y then  the  conditional  failure  race  of 

X given  Y > y decreases  as  y Increases,  which  Is  a form  of  positive 

dependence . 

Now  let  us  suppose  we  have  a sample 

(Xj^.Y^)  , <^n'V 

from  absolutely  continuous  bivariate  distribution  F on  [0,«}  x [0,w) 
Let 


^(1)  ^ ^(2)  ^ " ^(n) 


X-values  associated  with  Y 


■ *(n(i)) 

the  order 

^(1)  ■ 

1 of  subaamplas 

of  the 

^*(Tr(l)) 

^(iT(n))J 

*<it(n))^ 



^(Tr(n))^ 

n-1 

X 


^^<ir(n) 


} 
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Define 


i‘a)  ■ I + <n  - k - \ 0 < J < . - k 

0 <_  k ^ n - 1 

where  • • • • * ^ke  ordered  observations  In  x**”'*  • 

Now  plot 

j(f•''‘‘<l>•K)'  tlik<»-l) 

( O^j^n-k). 

We  call  this  the  empirical  conditional  scaled  bivariate  total  time  on 
test  plot  of  X given  Y . From  this  plot,  as  In  the  univariate  case,  we 
can  determine  the  behaviour  of  rj^(x,y)  . We  do  a similar  plot  with 
the  Y's  In  order  to  study  r2(xiv)  . 

Figure  5 shows  a bivariate  plot  based  on  running  and  down  times  of 
the  Yankee  Nuclear  Power  Plant.  We  present  the  conditional  plot  of  the 
down  times  given  running  times.  The  plot  Indicates  that  the  down  and 
running  times  are  positively  dependent  and  the  down  times  arc  DFR,  A 
Bivariate  Welbull  appears  to  be  a likely  model.  The  bivariate  case  is 
discussed  In  detail  In  [7]. 
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ABSTRACT 

When  there  are  available  several  point  estimators  of  component  or 
system  reliability,  It  would  be  of  Interest  to  compare  such  estimators 
through  some  "closeness  to  the  true  value  of  reliability"  criteria. 

Along  these  lines,  the  concept  of  Pitman-closeness  efficiency  Is 
Introduced.  Essentially,  when  comparing  two  estimators  of  reliability 
for  a given  situation,  Pitman-closeness  efficiency  gives  the  odds  In 
favor  of  one  of  the  estimators  being  closer  to  the  true  value  of 
reliability  than  Is  the  other.  Theory  Is  developed  which  provides  a 
straightforward  way  to  evaluate  this  measure  of  efficiency  under 
fairly  general  conditions  on  the  estimators.  Based  on  this  methodology, 
a comparison  Is  made  of  several  estimators  of  reliability  based  on 
Type  II  censored  data  from  the  one-parameter  exponential  failure  model. 


Key  words:  Pitman-closeness  efficiency;  Reliability  function; 
Exponential  failure  model;  Comparison  of  estimators. 


This  paper  was  presented  (under  a different  title)  at  the  23rd  Conference 
on  the  Design  of  Experiments  In  Army  Research,  Development  and  Testing. 


381 


PITMAN-CLOSENESS  EFFICIENCY  OF  ESTIMATORS 
OF  RELIABILITY  WITH  APPLICATION  TO  IHE 
EXPONENTIAL  FAILURE  MODEL 

Danny  Dyar  and  Jaroma  P.  Kaatlng 

1.  INTRODUCTION 

Point  aatlmatton  of  consonant  or  ayatam  rallablUty  at  tlna  t, 

R<t)  aProbCX  > t}  vhera  X la  tha  fallura  timat  ia  a fraquantly  oon- 
aldarad  problan.  Qanarally  apaaklngt  for  tha  oomnonly  aaaunad  fallura 
nodalBi  thara  ara  varioua  mathoda  hy  whldi  an  aatlnata  may  ba  obtalnad. 
Whan  thara  ara  avallabla  aavaral  point  aatlmatora  of  X(t),  tha  tradi- 
tional mathod  of  oon^arlng  aueh  aatlmatora  la  maan  aquarad  afflolanoy 
(l.a.f  tha  ratio  of  maan  aquarad  arrora).  Howavari  axeapt  for  a fax 
eaaaa  (e.g**  Zaoka  and  Evan  1966 » Slnha  1972 t axponantlal  fallura 
inodaK  Zaoka  and  Milton  1971 1 normal  fallura  modal)*  tha  datermlnatlon 
of  a oloaad-form  axpraaalon  for  maan  aquarad  arror  can  prova  to  ba  a 
difficult  talks  Furtharmora*  maan  aquarad  afflolanoy  oomparaa  tha 
ouarapa  parformanoa  of  an  aatimator  ralatlva  to  anothar}  oonaequantlyi 
ita  uaafulnaaa  ian't  alxaya  claar*  aapeolally  xhan  a alngla  astlroata 
la  to  ba  rnada.  Finally*  maan  aquarad  afflolanoy  can  be  mlaleadlng  and 
unrallabla  (aaa*  a.g. * Dyar*  Keating*  and  Hanalay  1977}  Box  and  Tlae 
1973*  p.  307). 

Wa*  therafora*  propoaa  anothar  maaaura  of  afflclancy  rootlvatad  by 
tha  folloxing  Idaa  dua  aaaantlally  to  Pitman  (1936).  Lat 
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« g^CXstfii),  1 a*  lt2t  ba  two  •ctlnators  of  rallablUty 
X(t)  ■ hCTfS)  baaad  on  a randon  aanpla  X of  alaa  n from  a glvm 

A 

failuva  modal  with  vaiknanm  paramatan  0.  Than  Xj^(t)  ia  aald  to  ba 

* 

a Pltman-eloaap  aatlmator  of  R(t)  than  la 

Ppob(|Rj<T)  - R(t)|  < “ »(»>!>  1 •* 

fbp  all  (T(9|n)  with  atrlot  Inaquallty  for  aona  (TQieg*ng)*  [Notat 
Undar  Rltman'a  original  oonoapti  Prab{)R^(t)  > R(t)|  < |R2(t)  - R(t)|} 
would  ba  a fiducial  probability  atatamant.  In  thla  papar*  howavar, 
Prob{|^(T)  - R(t)|  < |^<t)  - R(t)|)  la  datamlnad  In  tha  olaaaloal 
aanaa  and,  tbarafora,  haa  a ralatlva-fxaguancqr  Intarpratatlon.  3 Ha 
daflna  tha  Pitman-eloaanaaa  (PC)  a^RFlolanqr  of  ^(t)  ralatlva  to 
^(t)  aa 

A A 

ralf0ff.pQ(R2(t),R2(T)(T,e,n} 

Prob{|Rj(i)  - R(t>|  < iRgCr)  - R(t)|} 
Prob{|R2(T}  - R(t)|  < - R(t)|> 

Pltnan-oloaanaas  afflolanoy  givaa  tha  odda  that  ona  aatlmator  la 
oloaar  (In  abaoluta  valua)  to  tha  trua  valua  of  raliabllity  than  la 

A 

tha  othar  aatlmator.  For  axanpla.  If  ral.aff.pQ(R^(T),R2(T)t 

m 

T,0,n>  ■ 1.5,  than  tha  odda  ara  3 to  2 in  favor  of  R]^(t)  balng  oloaar 

A •A 

to  RCt)  than  la  Whan  ral.aff .pQ{R^(T),R2(T)|T,e,n}  > 1 

A 

fbr  aona  (tg,6||,n^),  wa  ahall  aay  that  R^(t)  la  mora  Pitman-oloaanaaa 
affloiant  than  R2(t)  at  (Tg,eg,ng).  If  ral.aff.pQ{R^(T)* 
R2(t)(T,B,n)  >,  1 for  all  (T,e,n)  with  a atrlot  inaquallty  for 


:‘W  'r='«f»''WriW.*^ 


A 

■otne  (To*  ^0*  >^)*  Pltman-closenMB  Inad- 

A 

odBsibl*  ralatlv*  to  Rj^(t), 

In  thiiB  paptri  w«  dovolop  thaoxy  which  provides  a straightforward 
way  to  evaluate  (1.2)  under  fairly  general  conditions  on  the  esti- 
nators.  Through  the  use  of  Pitnan-oloseness  efficiency • we  then 
oenpare  several  estimators  of  reliability  based  on  Type  XI  censored 
test  data  from  the  (one-parameter), exponential  failure  model.  The 
estimators  considered  are  based  on  (a)  the  method  of  maximum  likelihood » 
(b)  minimum  varisnoa  unblasadneBSt  and  (o)  the  mean  and  median  of 
Fraser's  structural  distribution  of  R(t). 

2.  THEORY  AND  METHODOLOGY 

A 

Let  R(T{n)  bs  an  estimator  of  reliability  R(t)  which  dspsnds  on 
(i)  the  sample  sise  n*  and  (li)  t and  the  life-test  data  only  through 

A 

a statistic  T whose  range  is  Z.  He  shall  aay  that  R(T(n)  is  a non- 
daortaaing  fult-rang§  Batimatpr  of  R(t)  if,  for  fixsd  n, 

(t)  R;  I C0,1]  (it  may  be  that  one  or  both  endpoints  of 
C0,ll  are  attained  only  by  considering  the  extended  range 
of  T)|  and 

(ii)  R(Tin)  la  a nondecreaslng  function  of  T on  I but  is  strictly 

A 

increasing  on  I'  C Z,  where  0 < R(T;n)  < 1 whenever  T 6 Z*. 

We  assume  Prob{T  €l'}  ^ o. 

A nondecreasing  full-range  estimator  of  R(  r)  is  necessarily  continuous 
on  Z. 


[( 
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„ .•'Spif-;«-'‘^‘’'  T!t^  - ‘MJ!-  .!««»«MV*«Rr-;«^1 


PtUnition  i,J  Let  R^(Tin),  i‘'l»2t  be  nondeoreaelng  full-range 
eatimatora  of  R(  t).  For  fixed  b,  x^  la  aald  to  be  a point  of  intov/- 
Motion  of  Rj^(Tjb)  and  R^CTjn)  provided 

(1)  R^(Xjtn)  ■ R2(X(,tn)j 

(11)  for  arbitrarily  email  a > o,  Rj_(x,)  ♦ c|n)  - RgCxj  i-  «jn) 

^ «% 

tnd  - «in)  - ^2^^©  " oppoilt*  signs « 

VtiitUtion  t,i  Let  R^CTtn)*  1*1|2»  be  nondeoreaelng  full-range 
eatimatora  of  R(t).  For  fixed  n»  la  aald  to  be  a m/Ctohing  point 
of  Rj^(Tin)  and  R2(Tin)  at  R(t)  provided  Rj^Cy^m)  + 

Note  that  x^  dapenda  only  on  the  aample  else  n;  y^^  dependa  on 
n aa  well  aa  R(t).  For  fixed  ni  let  X^C  I be  the  aet  of  pointa  for 

* A 

Whloh  Rj^(Tin)  and  R2(T|n)  are  not  both  aero  or  both  one»  When 

A A 

T oontinuoua  and  Inoreaaing,  thua  the 

awltohing  point  of  Rj^(Tjn)  and  R2(T}n)  at  R(t)  la  unique.  We  now 
prove  the  main  reault  of  thla  aeotion. 

A 

TfccoACW  f . 1 Let  Rj^(T;n)|  1*1 12(  bo  nondecr  laalng  full-range  eatl- 

matora  of  R(t).  In  « (aib}|  let  y^  be  the  unique  awltohing  point 
at  R(t)  and  auppoae  that  Rj^(tin)  ■ R2(tin)  only  at  a finite  number 
of  pointa  of  Interaeotion,  aay,  Xj^,....x^.  If  R^^Ctjn)  > RjCtjn)  when- 
ever te  (a,  mln(xj^,...,XN)),  then 

Prob(|Rj^(Tin)  - R(t)|  < |R2(Tin)  - R(t)|} 


C(N+l)/2] 


C*’*'ob{x 


(2k)  ^ ’’  ^ ’*(2k-t-l)^’ 


(2.1) 
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wh«M  ■ a.  x^j^j  X min  (y^,xj,,...,x^),.  . ., 

V*  ’‘(N+2)  ^ ^• 

PtiooH  If  y^»  Xj^  for  seme  1,  th«  rasult  follows  imcdlatoly  by 
taking  ■ X(j4.]^)  "Vo  * xb«ra  ther*  aro  oxxotly  j-1  of  th« 

Xj^'i  lose  than  yp, 

Attuma  that  y^  x^  for  any  1.  Cenaidar  ths  partition  of 
datarminad  by  tha  diatlnot  points  a » x^^^  < x^^^  < ...  < * 

’‘(Nt2)  * idanti^  thosa  opan  intarvala  Z„(j)  " ’^(j-fl)^ 

Buoh  that  whanavar  t € Ij|(j)t  tha  inaquality 

|Rj^(t}n)  - R(t)|  < iR^Ctin)  - R(t)|  (2.2) 

holds.  Whan  t e(x^gj«  x^^)»  (2.2)  holds  by  hypothaais.  Conaa- 
^uantlyi  tha  thaoram  will  follow  by  showing  that  (2.2)  holds  ovar 
axaotly  ona  of  two  adjaoant  opan  intarvala i aayi  X^(i)  ■ ’^(ifi)^ 

C(Ue  yp  i x^).  Whan  yp  ^ ’*(£)*  't*’*"  Rj^(t>n)  + R2(t;n)  > 2R(t)( 
thus  ^^(tin)  - R<t)  > R(t)  - R^(tin)»  whara  t e 1^(1)  ^ Ij^Cl+l). 

Suppoaa  Rj^(tin)  > l*2(ttn)t  t €1^(1).  Than  R^(t;n)  - R(t)  > 

R2(ttn)  - R(t)i  and  R(t)  - Rj^(tin)  < R2(t|n)  - R(t)  < Rj^(tin)  - R(t). 
Conaaquantly,  |R2(tjn)  - R(t)|  < |Rj^(tin)  - R(t)|  alnoa  R2(t(n)  > R(t). 
Furthamorsi  Rj^(tjn)  < R2(ttn)t  t € l^(l+l)i  hanoa  Rj^(tin)  - R(t)  < 
R2(tin)  - R(t).  It  follows  that  R(t)  - R2(t(n)  < Rj^(tjn)  - R(t)  < 
Rj2(t{n)  - R(t)  or,  aqulvalantly,  (Rj^(tin)  - R(t)|  < |R2(tin)  - R(t)1 
alnoa  R2(t|n)  > R(t).  On  tha  othar  hand,  if  Rj^(t{n)  •<  R2(t{n), 
t * '*^*’***  (2.2)  olaarly  holds  ovar  ^^(i)  ^ut  not  over  I^(l+1). 
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.•  9-  ■**-•*» v r 


A A 

Ocue  2t  ^0  -*‘(1+2)*  ^ ^ 2R(T)i 

thUB  R2(tin}  - R(t)  < RCt)  - Rj^(t;n),  where  t € U l^(lfl).  If 

R^(t;n)  > R^Ctin))  t elj^(l)»  then  R^(t;n)  - R(t)  > R^Cttn)  - R<t), 

Thue  R2(t|n)  • R(t)  < Rj^(t|n)  - R(t)  < R(t>  - R^Ctjn),  and 

|R^(ttn)  - R(t)|  < |R2(t;n)  > R(t)(  elnee  R(t)  > R^Cttn).  Moreover, 

Rj^(ttn)  < RjCttn),  t 6 hence  Rj^(t;n)  - R(t)  < R2(tin)  - R(t), 

Coniequently,  Rj^Cttn)  ~ R(t)  < R^Cttn)  - R(t)  < R(t)  - R^(ttn),  end 

|Rj<t|n)  - R(T)|  < iRj^(tsn)  - R(t)|  einoe  R(t)  > Rj^(tin).  On  the 

other  hend,  if  Rj^Ct^n)  < RjCtm),  t 6 then  (2.2)  cleerly  holds 

over  I.(ltl)  but  not  over  I.(i)* 
n n 

A A 

CdAC  3«  yp  ■ ’‘(i+D*  When  y^  ■ then  Rj^(tjn)  + RjCtm)  * 2R(t), 

t €l^(l)t  however,  R^Ctjn)  + R2(ttn)  > 2R(t)»  t ei^jd+l).  If 
Rj^(tin)  < R2(ttn),  t then  R^Ctm)  < R2(t|n),  t «l^d+l), 

end  (lee  Cues  1 end  2)  (2.2)  holds  over  Xj^dfl)  but  not  over  Zy,(0. 

If  R^^Ctin)  > RjCtin),  t 6 1^(1),  then  Rj^(tin)  > R2(t|n),  t «Ijjd+l), 
end  (see  Cssee  1 end  2)  (2,2)  holds  over  X^(i)  but  not  over  I^(lfl). 

The  proof  is  now  complete.  ■ 

If,  for  s given  n,  there  ere  no  points  of  intersection  of 
Rjj^CTin)  snd  RjCTin)  in  1^,  we  then  hsve  the  following 

^HetZaKH  ttl  Let  R^^CTin),  i"li2,  be  nondecressing  full-rsnge 
estlmstors  of  R( t) . Suppose  there  sre  no  points  of  Interseotion  of 
Rj^(Tin)  snd  Rj(T»n)  in  • (s,b).  If  Rj^(t|n)  * R^Ctin),  t «1„,  then 

Prob{|R^(T;n)  - R(t)|  < iRjCTm)  - R(t)|) 

f.  vr  n.  t 

■ Prob{s  < T < yp  }, 
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V. 


wh«r«  Yg  is  the  unique  switching  point  of  Rj(T}n)  snd  Rjdjn)  at  R(t). 

We  point  out,  In  pasting,  that  If  (in  the  theorem  as  well  as 

the  corollary)  R^^Cttn)  < RjCtjn)  whenever  t e(a,  min(>tj^ )«^)), 

then  Prob{|Rj^(Tjn)  - R<t)|  < (R2(T|n)  - R(t)|}  clearly  equals  one 
minus  the  right  hand  side  of  (2.1)  or,  in  the  case  of  the  corollary, 
(2.3). 

3.  ESTIMATORS  OP  RELIABILITY 

We  oonalder  a specif io  failure  model,  namely,  the  one-paramster 
eKponential.  Let  i^(2)  i*“  — ^(k)  ®*‘^**'*‘*  ^ 

failure  times  in  a random  sample  of  else  n (l.e.,  a Type  II  censored 
sample)  from  fjj(xi0)  ■ (1/0)  exp(-x/0),  x > 0.  The  reliability  st 
time  T > 0 is  given  by  R(t)  ■ Prob{X  > t)  ■ exp(>T/9).  The 
statistic 

T -n 

is  complete  sufficient  for  R(t).  Furthermore,  it  is  well  known 
(Epstein  snd  Sobel  19S4)  that  T has  a ganna  distribution  with  shape 
parameter  k and  scale  parameter  C-lnR(T)3"^.  We  point  out  that,  for 
a given  k,  the  distribution  of  T depends  on  t and  9 only  through  R(t). 
In  other  words,  we  need  only  speoify  the  value  of  R(t)  and  not  that 
of  T and  9 to  Index  the  distribution  of  T.  Consequently,  we  hence- 
forth write  R instead  of  R(t),  It  follows  that  2(-inR)T  has  a chl- 
squars  distribution  with  2k  degrees  of  freedom. 

By  the  invariance  property  of  maximum  likelihood  estimators,  the 
maximum  likelihood  estimator  (MLE)  of  R is 
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(3.2) 


R„j^j.(T;k)  = exp(-k/T),  T > 0,  k^  1. 

Batu  (1964)  has  ahown  that  tha  minimum  variance  unblaaed  eatlmator 
(MVUE)  of  R is 

Rjjyyg(T{k)  *0  , 0 < T £ 1 

« (1  - 1/T)^"^,  T > 1,  k > 1.  (3.3) 

We  noH  dlaouaa  two  other  eatimatora  of  R baaed  on  Fraaer'a 
(1968)  atruotural  Inference  (a  group>theoratlc  approach  to  Flahar'a 
fiducial  theory),  In  the  structural  approach • m In  the  Bayesian 
approach  I the  given  data  Induce  a probability  measure  (called  the 
atruotural  density)  on  the  parameter  space.  Unlike  the  Bayesian 
approach,  however,  this  is  achieved  by  assuming  the  existence  of  a 
atruotural  model  rather  than  a prior  probability  measure.  The 
atruotural  density  of  R,  given  T ■ t > o,  is  (Maxwell  1973) 

g(r|t)  ■ t’'(-lnr)’'"V“^/r(k),  0 < r < 1,  k > 1.  (3.4) 

Expression  (3.4)  Is  also  the  Bayesian  posterior  density  of  R under 
Jeffreys'  nonlnformative  prior  for  a scale  parameter  (see  Box  and 
Tlao  1973,  p.  44). 

Measures  of  central  tendency  of  the  structural  density  of  R 
would  seem  natural  choices  for  point  estimators  of  R.  Since 
E(R|t)  • ^^g(r|t)dr  » Cl  - l/(tn)3’*,  an  estimator  of  R based  on 
the  mean  (SMN)  of  the  structural  density  of  R is 

RgM^CTsk)  - Cl  - l/d+l)]*',  T > 0,  k > 1.  (3.5) 
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In  addition,  by  solving  the  equation  /^g(r|t)dr  = .5  for  m,  we  ob- 
tain an  estimator  of  R based  on  the  median  (SND)  of  the  structural 
density  of  R.  The  solution  is 

Rg„j,(Tik)  ■ exp(-m2^/2T),  T > 0,  k > 1 (3.6) 

where  is  the  median  of  a ohl-square  distribution  with  2k  degrees 
of  freedom.  For  specified  k,  the  value  of  m2j^  oan  be  found  in 

A 

Harter  (1964).  It  is  interesting  to  note  that  could  also 

be  obtained  by  a olassioal  argument.  Since  2(-lnR)T  has  a ohl-square 

A 

distribution  with  2k  degrees  of  freedom,  c 50  percent 

A 

lower  (upper)  confidence  bound  on  R.  Thus,  **  likely 

to  underestimate  as  to  overestimate  the  true  value  of  R.  In  this 

A 

sense,  sometimes  called  a median  unbiased  estimator 

(Lehmann  1959,  p.  83). 

A A A 

For  k >.  1,  we  note  that  Rj^j^d’jk),  Rs^(Tjk),  and  Rg^p(T>k)  are 

A 

each  nendeereasing  full-range  estimators  of  R.  Furthermore,  R^|jg(Ttk) 

A 

also  has  this  property  provided  k ^ 2.  When  k ° 1,  R^^g(T;l)  is  a 
zero-one  estimator  of  R and,  as  such,  is  of  no  practical  interest. 

A 

Henceforth,  any  dlsousalon  conceding  R^^(T;k)  will  assume  k >,  2. 

4.  FAIRWISE  COMPARISON  OF  ESTIMATORS 

The  theory  and  methodology  of  Section  2 will  now  be  applied  to 
the  estimators  of  Section  3. 

We  show  that  * single  point  of 
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intersect  Jon  in  Zj^  = (0,^*) , k ^ 2.  There  is  clearly  no  point  of 
intersection  in  (0,1].  Whan  t > 1*  R^Q^Cttk)  « 
equivalent  to 

h(t)  « t lnCt/(t-l)]  s k/(k-l).  (4.1) 

Since  e)(pCl/(t-l}]  > t/(t-l),  t > 1,  then  h'(t)  < Oj  thus,  h(t) 
ia  decreasing.  The  exiatanoe  of  a unique  root  for  (4.1)  follows 
since  1 < h(t)  < +••  and  h(t)  la  continuous.  By  Theorem  2.1, 

- *1  ‘ I • *•> 

* ProWO  < T < mln(>«j,y0))  + Prob{T  > max(Xoiyo)} 

■ x^l2(“lnR)  niln(j<Q,yo);  2k] 

+ 1 - x^C2(-lnR)  nax(xo,yo)(  2k],  (4.2) 

where  for  specified  k 2,  Xg  is  the  solution  to  exp(-k/Xg)  ■ 

(1  - l/Xg)^"^;  yg  is  the  solution  to  exp(-k/yg)  + (1  - I/Vq)^  ^ = 2R 
whenever  R >.5  exp(“k),  otherwise,  y^  * -k/ln{2R).  We  write 
X*( • jm)  as  the  chi-square  distribution  function  with  m degrees  of 
freedom.  When  k « 2,3,5,7,10,  and  20,  expression  (4.2)  has  been 
evaluated  nunierlc€d.ly  for  R ® 0(0.01)1.0.  Based  on  these  results, 
the  graphs  of  rel.eff.pg{R|^j^g(T;k),Rj|jyyj:,(Tik)iR,k)  versus  R 
are  given  in  Figure  A. 

4.2  ^E^Tik)  and  ^(,|j3^'*';k) 

There  are  no  points  of  Intersection  of  ^MD 

in  Ij^  a (0,-»»),  k ^ 1.  This  follows  since  the  median  ia  less  than 
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th«  MU  of  a ohl-square  distribution  (Groeneveld  and  Meeden  1977)  t 
oonsaquantly,  ^MijCtjk)  • sxp(-in2jj/2t)  > a*p(-k/t)  ■ ^j^Cttk), 
t > 0.  By  Corollary  2.1, 

f«»W|R8HD^T|k)  - R|  < !^(Tik)  - R|} 

■ Probft  < yj)  - ^C2(-lnR)yo  j 2k],  (4.3) 

whsra  for  spao.^ . jd  k ^ 1 ud  R,  y^  is  ths  solution  to  oxp(-m2j^/2y|j> 

+ aHp(-k/y^|)  e 

For  fixad  k 1,  * Rn^pCttk),  t > 0;  furthsrmors , 

RgKQ(Ttk)  undsrsstlmatss  R,  on  ths  avsrags,  50  psrosnt  of  ths  tins. 

It  follows  that  Rg|jQ(T|k)  is  olossr  to  ths  trus  valus  of  R than  is 

A 

RjOjjdik),  on  ths  avsrags,  at  Isut  SO  psrosnt  of  ths  tlM.  Zt  will 
bs  instruetivs  to  provs 

A 

ThiOfum  4,1  R^^g(T;k)  is  Pitnan-clossnsss  inadmissibls  rslatlvs 

to  Rg^(Tik). 

For  fixsd  k ud  R, 

Csxp(-k/yp)  sxp(-«i2jt/2yo)]** 

< CsxpC-k/y^)  + sxp(-m2^/2yg)]/2 

< sxp(-m2j^/2yp), 

alnos  ths  gsoMtrlc  moan  of  two  unsqusl  posltlvs  rsal  numbsrs  is  Isss 
thu  thoir  arithMtic  nsu.  By  sinplifioatlon, 

'"2k  ^ 2<-lnR)yo  * <">2k 

Hsnos,  for  k 1 and  0 < R < 1, 


*1 


. Jf.-  , .t 


' .5  < >^C2(-lnR)yoJ  2k]  < X^CCWj,^  ♦ 2k)/2;  2k] 

s;  /•l+ln2 

I < x*C(m2  ♦ 2)/2;  2]  a 1(1/2)  •xp(>x/2)dx  « .S71, 

I " ’'o 

I By  (4.3),  It  fbllom  that 

I 1 < ral>aff.pQ{Rglf0(T|k)*^j,(T(k)|R,k>  < 1.331.  ■ 

I 

r 

: A 

Fpob  tha  proof  of  Thaoram  4.1,  tha  odda  in  favor  of  Rg^(¥tk)  balng 
i eloaar  to  R than  la  R)|jj.(Tik)  ranga  from  allghtly  battar  than  avan 

to  approxlaataly  4 to  3.  Tha  axaot  axtant  of  thla  afflolanoy  oan 
ba  dataralnad  from  tha  grapha  (baaed  on  numarloal  evaluation  of 
I (4.3))  of  ral.aff.pQ{^l^(T{k),R|^|,(Ttk)tR,k}  varaua  R givan  in 

Figura  B. 

4.3  ^MD^Tik)  and  R„vuB^'*'*’‘^ 

Wa  uaa  an  argument  aimllar  to  that  of  Subaaotion  4.1  to  ahov 
that  Rgmj(Tjk)  and  Rj^Crjk)  have  a alngla  point  of  Intaraaotlon 
in  (1,+-).  When  t >*  1,  ^^pCtik)  » aqulvalant  to 

h(t)  a t lnCt/(t-l)]  a m2j^/2(k-l).  (4.4) 

Hotravar,  h(t)  la  continuoua  and  decreasing;  f urtharmot'a , 1 < h(t)  < 
Since  2(k-l}  la  tha  mode  of  a ohi-aquara  diatrlbution  with  2k  dagraaa 
of  fraadem  and  la  lass  than  (Oroanavnld  and  Meeden  1977),  it 
followa  that  a unique  root  for  (4.4)  axlsta.  By  Thaoram  2.1, 

I Prob{|5s^(T;k)  - R|  < l^(Tik)  - R|} 

* ■ x*C2(-lnR)  mln(Xg,yQ)t  2k] 

+ 1 - x*C2(-lnR)  tnax(xQ,yQ);  2k],  (4.5) 
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wh«r«  for  apoclfled  k ^ 2,  Xq  Is  ths  solution  to  sxp(-m 
• (1  - yj)  1b  thB  lolutlon  to  wcpt-iOgj^/ayo)  + (1  - l/y^)^"^ 

■ 2R  whsnsvsr  R >.S  sxp(>in2]^/2)t  otherwlss,  yg  • ••n2^/21n( 2R) . 

Tor  flxsd  k ^ 2|  Rg^^Ctik)  > R|^g(t;k)  if  and  only  If 
t C (0|Xq).  Conssqusntly,  s oonoluslen  similsr  to  that  which  Innsdl* 
•tsiy  ppsosdsd  Thsorsm  4*1  cannot  bs  mads.  Nsvsrthalaaa • wa  hava  tha 
folloifing 

A 

Thtofum  4,i  R|fyyj,(Tik)  is  Pitman-olosanasa  inadmiaaibla  ralatlva 

PAffoi  For  fixad  k|  lat  Xg  ba  tha  uniqua  point  of  Intaraaction  and 
Rg(k)  « axp(-mjj^/2xg)  ■ (1  - l/xg)*'"^.  Whan  R > RgCk),  than  1 < Xg  < yg» 
whara  yg  is  tha  switching  point  at  R*  Thua»  axp(«m2]^/2yg} 

< Caxp<-m2)/2yg)  + (1  - l/yg)*'"^]/2  or*  aquivalantly,  * 2(-lnR)yg. 
From  <4.5)i 

P«>b(|Rg(,o(T;k)  - R|  < l^gdjk)  - R|> 

« x*t2<-lnR)XQ}  2k3  + 1 - x*C2(-lnR)3^  » 2k] 

> x^t2(-lnR)Xgj  2k]  + 1 - X^<"'2k‘ 

Whan  R <Rjj(k),  than  < k|,  and  axp(-m2,/2yg)  > CaxpC-m  2k^^0  ^ 

A 

+ R^IV^ECyg |k)]/2.  Thus,  < 2(-lnR)yg.  From  (4.5), 

Prob{|Rgj,p(Tjk)  - r1  < |R,^(Tjk)  - R|) 

ti  X^C2(-lnR)yg>  2k]  + 1 - x^C2(-lnR)>«g » 2k] 

> X*<«>2x*  " 

A 

Whila  Thaoratn  4.2  says  that  i*  to  bs  prsfarrsd  evar 


a 'Ai 


i 

( : 

R»yyg(T|k)  from  a Pltman-cloaanasa  offiolancy  point  of  vlawt  tha 
axtant  of  this  efficiency  can  be  determined  from  the  grapha  (baaed 
on  numarloal  evaluation  of  (4.S))  of  rel.eff.p^Cl^igjCTtk), 

I A 

I R)(ym;(Ttk)|R|k>  varauB  R given  in  Figure  C. 

4.4  RgHD^Tjk)  and 

Wa  ahov  that  RggoCTtk)  and  * aingla  point  of 

I ^ 

intaraeotlon  in  k ^1*  When  t > 0, 

A 

■ Rgil^(ttk)  la  equivalent  to 

Kt)  * t ln(l  ♦ 1/t)  ■ m2j^/2k.  (4.6) 

Slnoe  axp(-x)  >1<-X|0<x<lt  A'(t)  ■ ln(l  + 1/t)  - l/(t+l)  > 0 
and  X(t)  in  oontinuoua  and  Inoreaaing.  For  fixed  k it  followa 

that  a unique  root  for  (4.6)  exieta  alnoe  0 < l(t)  < 1»  t > 0»  and 

"2k  * Denote  the  root  by  Xg»  Wien  t < Xqi  A(t)  < m2j^/2kt 

A A 

oonaequently,  Rg|^(tik)  > Rg|^(t{k).  By  Theorem  2.1, 

Prob{|^jpj(T;k)  - r|  < l^^ptTik)  - R(} 

« X^C2(>-lnR)  mln(xQ,yo)j  2k] 

+ 1 - x^C2(-lnR)  max(x5,yQ){  2k],  (4.7) 

where  for  apeoified  k 1 and  R,  Xg  la  the  aolution  to 

W S 

exp(  •mj2l^/2Xg ) • l1  - 1/(xq  + 1)]  j yo  ia  the  aolution  to  il 

exp(*m2|^/2yg)  + Cl  - l/(yo  + 1)]*^  * 2R.  The  grapha  (baaed  on  I 

numerical  evaluation  of  (4.7))  of  rel.eff,pp{R5jjjj(T|k),Rgj^(Tjk) jR,k> 

veraua  R are  given  in  Figure  D. 
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Thtrt  ar«  no  points  of  Intersootlon  of  R|^^y,(Ttk) 

In  Ij^  » (0,  -H*),  k ^ 1.  Sines  sxp(l/t)  > 1 ♦ l/ti  than 
Rjjj^j,(ttk)  ■ axp(-k/t)  < Cl  - l/(t+l)3^  ■ ^^(tik),  t > 0.  By 
Corollary  2.1| 

Probd^^CTjk)  -R|  < iR^gdjk)  -RD 

■ Prob{T  < y^j}  ■ x*C2(-lnR)y^»  2k],  (4,8) 

whsra  for  spsolflsd  k ^ 1 and  R,  y^  ia  the  solution  to  axp(>k/yg) 

+ Cl  - l/(y,+l)]’^  ■ 2R. 

ThwAon  4,3  Pltinan‘>Qlo8ansBB  afflolant  than 

R^(T(k)  If  and  only  If  R > R|}(k)  ■ •)(p(’'in2k^^’'^0^*  vhara  for 
apaolfisd  k i 1,  la  the  solution  to  a)v(-in2|^/2x^)  ■ (anpC-k/n^)  ♦ 
Cl  • 1/(>^*+1)3^)/2,  Furthsmors , .199  < RjCk)  < a'^ai  .368,  k ^ 1. 

PnooH  Tha  average  of  “ nondacraaslng  full- 

range  estimator  of  R.  We  first  show  that  4nd  CR|^g(Ttk)  t 

A 

RgM)((Ttk)]/2  have  a single  point  of  Intersection  in  (0,t«»).  When 
t >0,  Rgjjo^^sR)  * equivalent  to 

h(tik)  ■ t ln2  - t ln{exp(-k/t)  ■»■  Cl  - l/(t+l)]^) 

■ (4.9) 

For  fixed  k ^ 1,  h(tik)  la  continuous  and  Increasing  over  (0,t«). 
Furthermore,  llm  h(t(k)  ■ ki  thus  o < h(t{k)  < k,  t > 0.  Slnoe 

t4« 

< 2k,  the  existence  of  a unique  root,  say  x^,  for  (4.9)  la 
established.  Let  Rj(k)  ■ exp(-m2j^/2x‘ ).  Vfhen  t < xj,  h(t»k)  < 
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d-vn-aiL-unimm.i: 


thtui,  ^ ‘‘‘  *'®*' 

Itt  y^  b«  th«  iwltohlng  point  ot  ond 

Mhon  R > Rj(lt),  *g,|p(yo»J«)  * '•'  ^ 

OP,  oqulvalontly,  * 2(-lnR)y(|.  By  (4.8), 

Prob{|RgM^(T|k)  - r|  < |R^(T|k)  - r|} 

■ X^C2(-lnR)ygt  2k]  > X*(«2Jc*  * •®* 

Tht  lut  Inoquallty  la  ravartad  vhon  R < R((k). 

Pop  flxad  k >.1,  oxp(-2k/m2^}  ♦ Cnijjj/(«jj^+2)]*‘  < 2a"^j  thua, 

h(n2|^/2ik)  > n^/2,  Slnoa  h(tik)  la  Inopaaalng  In  t,  than 

*4  < «2k^2  R^(k)  < a'^H  .368,  k ^ 1.  Fupthannopa,  «2k/*>»o 

daopaaaaa  with  inoraaalng  k.  Tha  aolutlon  to  h(x^|l)  ■ IU2/2  la 

• .4286i  thua,  inln  R^Ck)  ■ R^(l)  ■ a>{pC-m2/2( .4289)] « .199.  ■ 

Although  tha  trua  valua  of  R la  unknown,  most  altuatlona  of 

Intapaat  would  praauppoaa  that  rtllablllty  la  at  laaat  37  papoant. 

« * 

In  thaaa  altuatlona,  Rg|g|(T(k)  would  ba  prafarrad  ovar  R^^^p(Ttk) 

from  a Pltman-oloaanaaa  afflolanoy  point  of  vlaw.  Tha  axtant  of 

thla  afflolanoy  may  be  datarmlnad  from  tha  graphs  (basad  on  numarloal 

avaluatlon  of  (4.8))  of  ral.aff.pQ{^^(T{k),R|^^j,(Tik)|R,k} 

varsus  R glvan  In  Flgura  E. 

A A 

4.8  and  ^ 

Per  apaolflad  k 3^  2,  wa  show  thara  is  axaotly  ona  point  of 
Intaraaotlon  of  Rg^^Tik)  and  R|fyyg(7tk)  In  (l,Ai).  Thara  la  olaarly 

A A 

no  point  of  Intaraaotlon  in  (0,1].  Whan  t > 1, 
la  aqulvalant  to 
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♦ 2)*'  ■ <M  + 1) 


I 


(4.10) 


2k-l 


whan  |i  ■■  t-1  > 0.  Equation  (4.10)  may  ba  wrlttan  aa  a polynomial 
aquation  In  p of  dagraa  2k-2|  i.a.» 


m 0.  (4.11) 


Lat  aj  ba  tha  ooaffioiant  of  For  j • lt...|kt 


- (*5'1  ■ ^ (*:i 


a^  • 2^ 


for  j ■ k+l|...,2k-lt 

•r  [ 1 1 • 

Than  a^  ■ 1(  aj  " h-lt  and  sinoa  03  ■ -(k-l),  it  followa  that 
a^  < 0|  j ■ 3|...(2k-l.  Conaaquantly I by  Daaoartas*  rula  of  aigns 
thara  it  axaetly  ona  peaitlva  root  for  (4.11) » and  thua  for  (4.10). 
By  Thaoram  2. I, 

Prob{|Rg^(T;k)  - r|  < |Rj,yyj.(Tjk)  - r|> 


■ X*C2(-lnR)  mln(x0,  yj)!  2k3 
t 1 - x*C2C-lnR)  max(X(,,  y^)!  ak], 


(4.12) 


whara  for  apeoifiad  k 12|  Xg  ia  tha  aolution  to  Cl  *■  l/(Xg+l)]  ■ 


(1  - 1/Xg)^"^l  Fo  aolution  to  [1  - l/(yu+l)]''  t (1  - l/Fij) 

■ 2R  whanavar  R > (1/2)'^^^,  otharwlee,  ■ Cl  - (2R)^'^^]“^  - 1, 

Aa  pravioualy  mantionad.  whan  tha  trua  valua  of  rallabllity  la 

A * 

at  laaat  37  paroant,  P*’*^**”:^^  ovar  RjjLu^Tik)  from 

a Pitman-oloaanaaa  affloianoy  point  of  vlaw.  A aimilar  atatamant  ean 


k“l 
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b«  Mdt  with  regard  to  end 

^laiSSSff,  When  R > e“^,  Rj^CTjk)  le  more  Pltmen-oloeeneee  efficient 

J»ssi  Pw  •p.em.J  k , J.  Irt  V”’ ■ * Ws'*’- 

irtiere  Kj  le  the  point  of  Intereeetlon  of  RgHH^****^ 

By  e Pede  retlonel  epproxlmetlon  of  Index  2 to  the  binomial  eerles 
(l.e.,  (l+x)"«  [2  + (n+l)x]/C2  - (n-l)x]|  Releton  196S,  pp»  378-280>i 
1 h - .St  thue,  Hj(k)  ^ e“^,  k > 2.  Let  be  the  ewltohing 
point  of  ^(j,(Ttk)  end  Rj|vuE(Ttk)  et  R.  Note  thet  when  R > e‘^,  then 

•fhe  everege  of  ^jj^(Tik)  end  R,|jyug(T|k)  le  e nondeoreaelng  full- 
*«n|e  eitimetor  of  R,  When  k 4,  there  ere  no  points  of  Intersection 
of  Rg„o(Ttk)  end  C^^(Tik)  + ^<T|k)]/2  in  (0,4«)  slnoe 

< t^^(ttk)  ♦ R^(ttk>3/2.  t > 0,  Thus,  Rg^pCyjtk)  < 

^ o***  equivalently,  > 2(-lnR)y0, 

k 1 4.  Prom  (4,12),  for  k .>  4 end  R > e“^, 

^Wl^jCTtk)  - R|  < |Rgjg,(Tik)  - R|) 

■ x*C2(-lnR)  mex(Xp,yg>i  2k3 

- x*r2(-lnR)  mln(Xp,yg)t  2k3 

■ x*Ca("lnR)yj{  2k3  - x*C2(-inR)xp t 2k3 

* 2k)  - x*C2(-lnR)Xji  2k3  <.5.  (4.13) 

Nhen  k ■ 2 or  3,  there  le  e unique  point  of  Interaaotion,  say 

“J*  t ^(Tik)3/2  in  (0,1).  However, 

the  inequality  Rg„j,(ttk)  < rRa^(t|k)  t R^(ttk)3/2  .till  holds 
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for  t > 1,  Since  x*  < 1 < then  (4.13)  holds  for  k > 2.  ■ 

Theorem  4.4  le  somewhat  conservative  In  the  following  sense.  For 
a given  k >,2,  it  is  quits  likely  that  Is  more  Pitman- 

A 

eloseness  efficient  than  values  of  R considerably 

leas  than  .368.  The  extent  of  this  efflolsney  is  shown  in  the 

A 

graphs  (based  on  numerical  evaluation  of  (4.12))of  rel.sff.p^{Rgll^(T|k)t 

A 

UnvUE^lk)!  Rik}  versus  R given  in  Figure  F. 

5.  DISCUSSION  OF  RESULTS 

Pltman-oloaenesB  efflolenoy  is  inherently  dependent  (i^on  the 
probability  that  an  estimator  undersstlmatss  (or  overestimates)  the 
true  value  of  the  parametria  function  being  estimated.  Slnoe 
Prob{Rjjj^g(Tik)  < R)  ■ x^(2k|2k)  > .S»  k >.  1»  then  tends 

A 

to  underestimate  R.  On  the  other  hand,  R^^£(T;k)  tends  to  over- 
estimate R whenever  R > e**^*  .388.  To  see  that  this  is  so»  first 
note  that  mjj^  > 2k  - .7,  k >.1$  thus  Cm2j^/(m2j^  - 2)]*'“^  <. 

tl  - l/(k-1.36)]^’^  < e or,  equivalently,  2(1  - expC-l/(k-l)!l)"^ 

< m^j^.  For  fixed  k,  q(Rik)  " 2(“lnR)Cl  - , j,. 

oMAsing  funotlon  of  R.  Thusi  for  flxod  k ^ 2 and  R > a » 
q(R(k)  < q(e  |k)  ■ 2(l  - expC-l/(k-l)]}  < m^j^.  The  result  now 

follows  since  Prob{Rjjyyjj(Tik)  > R>  ■ 1 - X*Cq(Rik)j  2k].  Moreover, 

-1  * 

as  R decreases  from  e to  0,  both  R|^yyg(Tik)  as  wall  as,  of  course, 

A 

R„,^(Ttk)  tend  to  underestimate  R.  Generally  speaking,  from  Figure 

A 

A,  Rm^p(Ttk)  is  more  Pltman-oloseness  efficient  than  R^fym,(Tik).  For 
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low  to  moderate  rellabllltyi  the  odde  In  favor  of  being 

closer  to  the  true  value  of  R are  quite  high.  For  high  reliability* 
the  odds  atill  favor  slishtly  better  than 

avan.  This  result  la  sonawhat  oenalstent  with  a maan  squared  efflolenoy 
oonparlaon  given  by  Zaoks  and  Even  (1966,  Flgtire  3).  They  show 
graphically  that  squared  efficient  than 

over  the  Interval  .03  < R <,61  (n  * k « 4,8),  Maximum  mean  squared 

A 

efficiency  la  reached  at  approximately  R ■ .22,  And  although 

A 

la  more  maan  squared  efficient  than  R » ‘Ol*  the 

mean  squared  error  of  R|(y|jg(Tik)  Is  only  slightly  leas  than  that  of 

^E<Ttk). 

When  R > 0“^,  Fraser's  structural  Inferenos  provides  estimatora 

which  fall  between  the  maximum  likelihood  estimator  (which  tends  to 

underestimate  R)  and  the  minimum  variance  unbiased  estimator  (which 

“1 

tends  to  overestimate  R).  In  other  words,  when  R > e (i.e,, 
t £ k - .6),  either  (a)  ^j^^(tjk)  < Rgug^tik)  < Rginjj^tjk)  < 

the  latter  Inequality,  I.e,,  (b),  that  Is  true  If  reliability  Is  high. 
When  R « s**^,  the  structural  estimators  give  estimates  of  R which 
are  larger  than  those  given  by  the  maximum  likelihood  or  minimum 
variance  unbiased  estimators  both  of  which  tend  to  underestimate  R, 

A A 

In  any  event,  by  Theorems  4.1  and  4.2,  R^p j^(T;k)  and  RfiivuE^^^^^ 

A 

are  both  Fitman'cloeenass  inadmissible  relative  to 

sequently,  from  a Pitman-closeness  efficiency  point  of  view,  the 

A A A 

preference  among  the  estimators  Rfit^g^Tik), 
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A 

Rg|i|^(T:k)  reduces  to  comparing  only  the  latter  two  except  In  the 
following  relatively  minor  situation.  When  k b 2 and  .17  < R < .22» 
more  Pitman-oloaeneas  efficient  than  ^jj£,(Tjk)s  in  tum» 
^SHD^******^  !•  wore  Pitman-cloaenasB  efficient  than  i^g(T|k)|  yet, 
R^(T}k)  is  more  Pitman-oloaeneea  efficient  than  ^^(T(k).  A 
similar  atatement  can  be  made  whenk  « 3 and  .20  < R < .22.  fortu- 
nately, thia  circular  paradox  vanishes  when  k > 4.  It  is  beoauna 
of  this  apparetit  absence  of  a general  transitivity  property  that 
pairwise  comparisons  of  the  estimators  were  made  in  Section  4. 

A A 

For  fixed  k,  a preference  set  for  Rg„o(T»k)  (Rgjgj(T»k)>iB  the 
aubinterval(s)  of  the  R-intsrval,  l.e.,  [0,1],  over  which 
(Rgjg|(Ttk))  is  more  Pitmen -closeness  efficient  than  ^jg^CT^k) 

Preference  sets  as  well  as  the  spsoifio  odds  in  favor 
of  being  closer  to  the  true  value  of  reliability  may  be  detensined 
from  Figure  D.  Regardless  of  the  number  of  failures,  i* 

preferred  when  xniliability  is  high  or  low.  The  odds  are  slightly 
better  than  even,  in  this  case,  that  will  be  closer  to 

the  true  value  of  reliability.  On  the  other  hand,  regardless  of  the 
number  of  failures,  Rg|^(T{k)  is  preferred  for  the  middle  values  of 
reliability,  i.«.,  the  component  is  about  as  reliable  as  it  is  unre- 
liable. In  this  case,  the  odds  favoring  KgiigfCl'sh)  being  closer  to 
the  true  value  of  reliability  are,  generally  speaking,  quite  high. 


402 


FIGURE  A.  EFFICIENCV  OF  KAXIflUR  LIICELIHOOD 
RELATIUE  TO  RINinUTI  UARIANCE 


FIGURE  B.  EFFICIENCV  OF  STRUCTURAL  tlEDIAN 
RELATIUE  TO  flAXinun  LIKELIHOOD 


RELIABILITY 


FIGURE  C.  EFFICIENCY  OF  ETRUCTURM.  HEOIAN 
RELATIUE  TO  HININUn  UARIANCE 


RELIABILITY 


FIGURE  D.  EFFICIEKCV  OF  STRUCTURAL 
RELATZUE  TO  STRUCTURAL  MEAN 
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